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ABSTRACT 

The purpose of this Thesis is to examine the Department of Defense Preaward Survey process and 
assess its effectiveness as a source selection and risk management tool. This is accomplished by 
identifying the primary causes of negative Preaward Survey recommendations, determining how Procuring 
Contracting Officers use this information in source selection and examining the Department of Defense’s 
experience with contracts awarded in the face of a Negative Preaward Survey. 

The data for this research were gathered by reviewing Preaward Survey files and interviewing contract 
administration office personnel at Defense Contract Management Command (DCMC) and Defense 
Logistics Agency Headquarters. Personnel from various buying activities throughout the Department of 
Defense were also interviewed. 

This thesis concludes that the Department of Defense Preaward Survey process is operating at a 
reasonable level of efficiency and effectiveness, but that improvements could be made to its use as a source 
selection and risk management tool. This conclusion is based upon findings that Preaward Survey 
information is being utilized effectively for contractor responsibility determinations. However, it was 
noted that Preaward Surveys are not always useful to facilitate source selection decisions and that Preaward 
Surveys are not being fully utilized as a risk management tool during preaward and post-award contract 
management. In addition, this thesis presents ten recommendations for further improvements to the 


Preaward Survey process. 





TABLE OF CONTENTS 


I. INTRODUCTION -------------------------------- 22-22-20 nn nn nnn nnn nnn nnn n nn nn nnn nnn n nnn nn nnn nn ene ne-en == eee 
A. GENERAL ----------------722--2----n 2-22 n nnn nn nn anne nnn nnn nnn n nnn nnn nance nner n nnn enneen nnn nnneeeeenennne= 
B. OBJECTIVE OF STUDY ------------------------------------- =n nnn nnn nn nnn none nnn nn nnn nn - eno ------------ 
C. THESIS SCOPE ----------------------------- 22-22-2222 nnn nn nnn nnn nn nn nnn wenn nn nen n ne ennen nanan nnn enen eens 
D. METHODOLOGY ----------------------- 222-222 n nn nnn nnnn nnn anne nnn nn nn nnn n nnn ene n ence nnn nnn ence nnen= 
E. ASSUMPTIONS AND LIMITATIONS ----------------------------2--2 2222 enone oon enn e nen e---e nn eee -- 
F. DEFINITIONS OF TERMS. -----------------------2------------- 2-2 e enn nnn nnn ee nenn nn nn nen en ennneennn=- 
G. THESIS ORGANIZATION ----------------2-------- 222-2 - nnn nnn nen n nn nnn nn nance nnn nnn nnn ennnnnn nen en ne 
Il. BACKGROUND --------rer-----2nnnnnnnnnnn nnn nnn nnnn nnn ance cen nn nnn nnn nen nn nn nn nee n en nnn enn nene nen nnn nnnnene= 
A. PURPOSE OF PREAWARD SURVEYS ------------------------------- 2222-00-02 nn nnn nner en anne ee eee 


B. DEFENSE CONTRACT MANAGEMENT COMMAND ORGANIZATION 
RIND YIU SST (IN fe seer nee ne ee 


ee PRE AWARD SURWMENEPRO GED URES Wiis sac 
De RESPONSE! T INMIDEME R NINA TION ete ee ee 
Tal, CABG a RAUL) 004 S84) 01 1 C)) 30 ee en Ree ener dene 


F. SMALL BUSINESS ADMINISTRATION CERTIFICATE OF COMPETENCY 
BR OG RUM (oxcesen ences se eee es nn = ee cee 


(OES INN RAV ssa anc ee nnn 


Ee oof GTRIINTEAN gl ca ar ec NR cr 


B. DEFENSE CONTRACT MANAGEMENT DISTRICT PREAWARD SURVEY 
SPN TEST 0) (Shs eee ee 21S ne 


C. DEFENSE CONTRACT MANAGEMENT COMMAND PREAWARD SURVEY 


[een AR ane p 


vil 


D. CONTRACT ADMINISTRATION OFFICE PERSONNEL INTERVIEWS -------------------- 43 


E. PROCURING ACTIVITY PERSONNEL INTERVIEWS ----------------------------------------- 57 
IV. DATA ANALYSIS AND INTERPRETATION --------------------n 20-2 on nnn nnn nena none nnn nanan nnn nee 73 
A. GENERAL  -------------------2----2 222-22 ------ === =n nnn nnn nnn nnn nn nnn nnn nnn nnn nn nnn nee enn nnn n nn nn eee 73 


B. ANALYSIS OF PRIMARY CAUSES OF NEGATIVE PREAWARD SURVEY 
fails @ OM INIEN TD AMON Sih co cece as oe ot ea ee ee 74 


C. ANALYSIS OF REASONS WHY PROCURING CONTRACTING OFFICERS 
OVERRIDE NEGATIVE PREAQVARD SURVEY. 6 22ee e- cacteteererne ee 81 


D. ANALYSIS OF DOD’S EXPERIENCE WITH CONTRACTS AWARDED IN 


THE FACE OF A NEGATIVE PREAWARD SURVEY ----—__ as 91 
E. ALTERNATIVES FOR IMPROVING PREA WARD SURVEYS AS A SOURCE 

SELECTION! A NIDIRISK. Win a Gir VEIN ai@) Olesen 94 

BES GUNG awn a Se ee ec a nee ee ae eee 116 

Vee CONGEUSIONS ANDIR EGOMIMENID AMON S .--2--c240.-----co--2-06se0---5-25-onc-s 7a eee eeewee 117 

Ay (CONCUUS OS a ee ee eae a 117 

Be UE CG VI EIV ESV AUTH NS ere cc ee Se cc oe See aeceneoee 122 

G) ANSWERS TO THEVRESEAR CH QUESTIONS —=--------—---02-=---- commenter ene canee 125 

DAJAREAS OF FUR SHER RESEARCH —-----:--2teeeeeee coun <n  __ 129 


APPENDIX A INTERVIEW QUESTIONS FOR CONTRACT ADMINISTRATION OFFICE 


BESS ©) NE lee ne ee 130 
APPENDIX B INTERVIEW QUESTIONS FOR PROCURING ACTIVITY PERSONNEL ------===-- 135 
APPENDBSC. GEOSSARWZOR ACRONYMS =---scscsscrcc occ nsae cece eenee: oeeeeen nec nnaas 140 
GEIB @IREDIR ERERENCES steereceseee cies cetca ts costar este cekict mance ees ee 141 
ENITIAL DISTRIBUTION LIST --—---------—----- rr rr 149 


Vill 


I. INTRODUCTION 


A. GENERAL 

Procuring Contracting Officers (PCOs) have a fiduciary responsibility to act in the Government’s best 
interest when awarding contracts to defense contractors. However, before awarding a contract, the PCO 
must make a determination that the prospective contractor has the necessary management, technical 
expertise, facilities, and financial strength to perform the contract. This determination is a critical step in 
the source selection process. 

Sound business practice dictates that the Government conduct business only with capable contractors. 
Award of contracts to non-responsible contractors could affect mission readiness if the contractor fails to 
deliver, fails to meet delivery schedules, or fails to meet requisite quality or performance requirements. In 
the event a contractor defaults on a contract, the procuring activity may have lost valuable time in fulfilling 
a Military Service requirement. Additionally, selecting an unqualified contractor may be costly. There 
may be significant financial and manpower costs associated with lost time and misdirected effort. Re- 
procurement costs may be higher than originally planned and budgeted, especially if accelerated delivery is 
required to make up for lost time. If the original contract 1s terminated due to contractor default, the 
Government may have to expend an enormous amount of resources to administer contract close-out and/or 
settle associated claims. 

Given the above, it is clearly in the Government’s best interest to award contracts only to responsible 
contractors. One tool to assist the contracting officer in making a responsibility determination is the 
Preaward Survey. When a PCO does not have sufficient knowledge of a potential contractor’s capabilities, 
he or she can request a Defense Contract Management Command (DCMC) conduct an on-site or desk 
survey to assess whether the prospective contractor has the requisite capability to satisfactorily complete 
the contract. Depending upon the information needed, the PCO may ask DCMC to assess up to twelve 


areas of a contractor’s operation. Potential review areas are grouped into major and minor areas. The 


major areas which may be reviewed during a Preaward Survey include: technical capability, production 
capability, quality assurance capability, financial capability and accounting system adequacy. Minor areas 
which may be reviewed include: Government property control, transportation, packaging, security 
clearance, plant safety, environmental/energy considerations, flight operations/flight safety and any other 
factor requested by the PCO. 

Results of the Preaward Survey, along with an award or no award recommendation, are forwarded to 
the PCO. Preaward Survey recommendations are not binding on the PCO. The PCO is responsible for 
making the responsibility determination and may decide to award a contract in the face of a negative 
Preaward Survey. Conversely, the PCO may decide not to award a contract to a contractor who received a 
positive Preaward Survey. 

Preaward Surveys are an integral part of the Government acquisition process, especially when the 
PCO has little or no knowledge about a prospective contractor’s capabilities. Preaward Surveys provide the 
PCO with sufficient, accurate and timely information about a prospective contractor’s capabilities. Often 
time, the Preaward Survey findings may be the only information available to assess the contractor’s 
capabilities and likelihood of abiding by contract terms and conditions. In this case, Preaward Survey 
information becomes an even more critical component of the source selection process. 

Knowledge of a contractor’s capabilities, strengths and weaknesses facilitates sound source selection 
decisions. If used properly, this Preaward Survey information can also be used to lower total acquisition 
costs and reduce government risk during the post award surveillance and administration phase of the 
contract. [Ref. 34] This information can be used by both the PCO and Contract Administration Office 
(CAO) to tailor post-award surveillance plans based upon the contractor’s capabilities and recent past 
performance on similar contracts. If a contractor has recently performed satisfactorily on similar 
government contracts, the PCO and CAO may elect to have no or minimal surveillance. Alternatively, if 
the contractor has a marginal record or has had specific problems in the past, the PCO and CAO may elect 
an increased level of monitoring or tailored surveillance of high risk areas. Used in this manner, Preaward 


Survey information can be a valuable tool in deciding what level of contract oversight is required to ensure 


satisfactory contract performance. In addition to aiding the source selection process, the above examples 
indicate that Preaward Survey information can be used to reduce contract performance risk and contract 
administration costs. 
B. OJECTIVE OF STUDY 

The purpose of this thesis is to examine the Department of Defense’s Preaward Survey process and 
assess its effectiveness as a source selection and risk management tool. This assessment will be based upon 
research data gathered to answer the following primary research question: What is DOD’s experience with 
contracts awarded in the face of a negative Preaward Survey recommendation and how might this 
information be used to improve the source selection process? Subsidiary research questions in support of 
answering the primary research question are as follows: 

1. What is a Preaward Survey and how are Preaward Surveys used as a source selection tool? 


2. What is a Negative Preaward Survey and what are the various DOD policies with respect to 
awarding a contract in the face of a Negative Preaward Survey? 


3. What are the relative percentages of positive and negative Preaward Survey recommendations? 


4. What are the primary causes of negative Preaward Survey recommendations and what impact do 
they have on contract performance? 


5. How closely do the reasons for contract performance failures line up with associated negative 
Preaward Survey recommendations? 


6. Do any of the reasons for awarding a contract in the face of a negative Preaward Survey play a 
role in a contractor’s satisfactory or unsatisfactory performance? 


7. What changes to the use of Preaward Surveys are needed to enhance it as a part of the source 
selection process? 


The overall objective of this thesis is to assess the use of negative Preaward Survey information in the 
acquisition process and validate its usefulness as a source selection and risk management tool. This 
information will help procuring and CAO personnel improve their effectiveness during source selection. 
Additionally, this information will improve the effectiveness of CAO personnel when administering and 
monitoring contracts awarded to marginal contractors (i.e., contracts awarded in the face of a negative 


Preaward Survey recommendation). 


C. THESIS SCOPE 

This thesis will provide an understanding of the Preaward Survey process, determine the major causes 
of negative Preaward Survey recommendations, and assess the effectiveness of negative Preaward Survey 
information as a source selection and risk management tool. 

The primary purpose of this thesis is to examine DOD’s experience with contracts awarded in the face 
of a negative Preaward Survey and assess how this information might be used to improve the acquisition 
process. This study will not address the validity of Preaward Survey requests or the effectiveness of 
methods used to perform Preaward Surveys. Additionally, this thesis will not investigate the effectiveness 
of the Small Business Administration’s (SBA’s) Certificate of Competency (COC) program. 

D. METHODOLOGY 

The following provides a brief overview of the methodology used to gather data for this study. Data 
presentation and a more detailed description of the study’s methodology can be found in Chapter III. 

Data will be collected from various Government and literary publications, Defense Contract 
Management Command files and interviews with CAO and procuring activity personnel. General and 
background information about DOD Preaward Survey processes and policies will be gathered from various 
Government publications, Internet “Home Pages” and literary publications. DCMD files will be used as a 
source to determine the relative percentages of positive and negative Preaward Survey recommendations 
made throughout the DCMC. Government Preaward Survey files located at DCMC Philadelphia will be 
used as a source to determine the major causes of negative Preaward Surveys. Interviews with procuring 
activity and CAO personnel will be used as a source to assess the effectiveness of Negative Preaward 
Surveys as a source selection and risk management tool. Specific interview questions used during this 
study are contained in Appendix A and Appendix B. 

E. ASSUMPTIONS AND LIMITATIONS 

The Defense Contract Management Command (DCMC) performs the most Preaward Surveys within 

the Department of Defense (DOD) and, as such, is considered to be the best source for Preaward Survey 


data. Defense Contract Management Command Philadelphia is a typical DCMC within DLA and their 


preaward survey records provide a representative cross section of the preaward surveys performed 
throughout DCMC. 

Defense contractors within the DCMC Philadelphia area provide a representative cross section of 
defense contractor capabilities within DCMC’s total area of responsibility. As such, contractor 
performance within this area of the country is typical of contractor performance found across DCMC’s 
other geographic areas of responsibility. 

Although this study attempted to get a cross section of data from all Military Services, this study is 
limited to data collected from Department of the Navy, Department of the Army and Defense Logistics 
Agency sources. Internal Department of the Air Force regulations precluded the collection of data from 
U.S. Air Force Procuring Activities without specific review and consent from the Department of the Air 
Force. This approval was not obtained in time to include U.S. Air Force data in this study. 

This study contains data collected during CAO and Procuring Activity personnel interviews. Several 
of the interview questions used during this research asked DOD personnel if they followed established 
Preaward Survey procedures and/or regulations. It is possible that some the responses to these questions 
were guarded or couched in terms of what these personnel were supposed to do vice what they actually 
practiced. No attempt was made by this study to verify the authenticity of specific responses. Therefore, 
this study may be limited somewhat by subjective opinions and comments rendered during personnel 
interviews. 

In addition to the above limitations, this study was limited by the availability and quality of various 
Government files and records. Preaward Survey files from 1995 were selected to facilitate analysis of 
DOD’s experience with contracts awarded in the face of a Negative Preaward Survey. In a few cases, the 
researcher could not gather data from Procuring Activities and DCMC due to missing or closed-out 
contract files. 

F. DEFINITION OF TERMS 
Administrative Contracting Officer (ACO): Warranted Government contracting officer responsible for 


post award contract administration, as defined in FAR Part 42. 


Certificate of Competency: Certificate issued by the SBA stating that the holder is responsible, as 
defined in FAR subpart 19.104, for the purpose of receiving and performing a specific Government 
contract. 

Contract Administration Office (CAO): Government office responsible for the administration of 
contracts awarded to contractors in their geographic area of responsibility. 

Defense Contract Management Command (DCMC): DOD activity under DLA responsible for 
providing contract management services in support of the Army, Navy, and Air Force, as well as DLA 
buying activities, the National Aeronautics and Space Administration, other Government agencies, and 
foreign organizations. Typical DCMC responsibilities and services include: planning and acquisition, 
contractor capability and proposal review, program and technical support, contractor in-plant production 
surveillance and quality assurance, contractor property management surveys, negotiation of contract 
modifications, and contract close-out. 

Defense Logistics Agency (DLA): DOD agency responsible for providing material support and 
contract administration services worldwide, primarily for military forces and other selected Federal 
activities. The broad DLA mission areas include: Supply Management, Material Warehousing and 
Distribution, Reutilization and Marketing, Logistics Information Management Support and Contract 
Management (also known as acquisition). 

Preaward Survev (PAS): An evaluation by a surveying activity of a prospective contractor’s capability 
to perform a proposed contract. PAS recommendations help the PCO make a responsibility determination 
and facilitates sound source selection decisions. 

Preaward Survey Manager (PASM): The individual within the DCMC who is responsible for 
managing, planning and coordinating Preaward Surveys within the cognizance of a particular CAO. 

Preaward Survey Review Board (PASRB): An advisory board within a DCMC responsible for 
reviewing Preaward Surveys and making recommendations to the PASM. The board is usually chaired by 
the Chief or Deputy Chief of the DCMC’s Contract Management Division and includes highly qualified 


senior representatives having industrial support, quality assurance and financial analysis expertise. 


Process Oriented Contract Administration Services (PROCAS): Contract administration service 


provided by DCMC which takes a proactive, quality oriented approach to administering contracts and 
strives to achieve cost saving enhancements on existing contracts. Instead of waiting to inspect an end 
product, PROCAS emphasizes prevention of problems and ee improvement from the very early 
stages of product development and manufacture. 

Procuring Activity: A component of an executive agency having a significant acquisition function and 
designated as such by the head of the agency. Synonymous with “Contracting Activity” and “Buying 
Activity”. 

Procuring Contracting Officer (PCO): Warranted individual at a procuring activity authorized to 
commit funds and resources on behalf of the Government. Prior to awarding a contract, this individual is 
responsible for ensuring the prospective contractor is a responsible contractor under the definition of FAR 
Subpart 9.104. 

Negative Preaward Survey: A report issued by a DCMC detailing a prospective Government 
contractor’s lack of expertise or capabilities to perform a proposed contract. As a result of negative 
findings in one or multiple areas, the report forwards a formal “No Award” recommendation to the PCO 
who requested the survey. 

Responsible Contractor: A contractor that meets the standards contained in part 9.104 of the Federal 
Acquisition Regulation. For example, a contractor must have a satisfactory record of integrity and business 
ethics, adequate experience and skills, and adequate financial and production resources (or the ability to 
obtain them) to perform the contract. 

Small Business Administration: Government activity responsible for promoting and assisting small 
business development. SBA has a network of nearly 100 field offices, over 13,000 volunteer counselors 
and more than 950 Small Business Development Centers. 

G. THESIS ORGANIZATION 
This thesis consists of five chapters. Chapter I provides a general introduction to the study’s objective, 


scope, assumptions and organization. Chapter II provides background information on DCMC’s 


organizational structure and the Preaward Survey process. Chapter III discusses the methodology used to 
gather data and presents the data gathered from Government files and procuring activity and CAO 
personnel interviews. Chapter IV provides an analysis of the data presented in the previous chapter. The 
thesis’ summary, conclusions and recommendations, as well as recommended areas of further research, are 
presented in Chapter V. Appendices A and B contain respective interview questions for CAO personnel 
and procuring activity personnel used during the conduct of this study. Appendix C contains a glossary of 


acronyms and Appendix D contains a list of selected references used in this study. 


Il. BACKGROUND 


A. PURPOSE OF PREAWARD SURVEYS 

The Armed Forces of the United States rely heavily upon the business community for the goods and 
services it needs to perform its mission. The types of products and services required by the Services range 
from relatively simple products, such as a nuts and bolts, to highly complex weapon systems, such as 
sophisticated fighter jets and aircraft carriers. In all cases, the products and services must be delivered on 
time, perform as specified and meet certain quality standards so as not to jeopardize the Department of 
Defense’s (DOD’s) mission readiness. This is especially true when products and services are procured to 
support critical applications. In these cases, the successful outcome of a mission or personnel safety may 
be at stake. 

Prior to contracting for products and services, Government policy dictates that DOD Procuring 
Activities evaluate a prospective contractor’s competence, capability and responsibility to perform the 
proposed contract [Ref. 115:p. 9-3]. The purpose of this policy is to assure the delivery of quality products 
and services at fair and reasonable prices. Within the Government, Procuring Contracting Officers 
(PCOs) are empowered to obligate funds and resources on behalf of the Government. The PCO also has 
the responsibility and authority to determine whether a prospective contractor is responsible to perform the 
proposed contract. 

To facilitate a responsibility determination, the PCO may use information on hand at the Procuring 
Activity or from outside sources, such as other Government activities and/or commercially available 
sources. If sufficient information about a contractor’s capabilities are not available, the PCO can request 
the Defense Contract Management Command (DCMC) to perform a Preaward Survey. The primary 
purpose of a Preaward Survey is to assess a contractor’s capabilities to perform on a particular contract. 
The Preaward Survey can assess virtually every facet of a contractor’s business or it can be tailored to the 


specific needs of a PCO. Typically, the major factors evaluated during a Preaward Survey include 


technical, production, quality assurance, and financial capabilities. Upon completion of the survey, a 
Preaward Survey report is forwarded to the PCO. This report contains detailed findings on all capability 
factors evaluated and contains an “Award” or “No Award” recommendation. The actual award decision 
and authority resides with the PCO. 

B. DEFENSE CONTRACT MANAGEMENT COMMAND ORGANIZATION AND MISSION 

As discussed above, the Preaward Survey process performs a valuable role in the acquisition process. 
The majority of Preaward Surveys performed within DOD are conducted by Contract Administration 
Offices (CAQOs) under the cognizance of the Defense Logistic Agency’s (DLA’s) Defense Contract 
Management Command (DCMC). The following provides background information on DCMC’s 
organization, mission and responsibilities. 

The Defense Contract Management Command (DCMC) is a major element of DLA, with headquarters 
located in Fort Belvoir, Virginia. DCMC’s mission is to provide worldwide Contract Administration 
Services (CAS) in support of the Military Services, National Aeronautics and Space Administration 
(NASA) and other Federal Agencies. In addition to providing CAS to domestic customers, DCMC 
provides services for foreign governments and international organizations. DCMC provides customers 
with a full range of preaward contract management and post-award contract management services. 
Specific services include: acquisition planning , contractor capability reviews (Preaward Surveys), 
contractor property management reviews, proposal reviews, program support, technical support, in-plant 
production surveillance, quality assurance, negotiation of contract modifications and contract close-out. 
[Ref. 114:pp. 6-7] 

DCMC is organized into three Defense Contract Management Districts (DCMDs), with each District 
responsible for providing contract management services within a geographic area of responsibility. 
DCMD East, with headquarters in Boston Massachusetts, is responsible for managing contracts within the 
eastern half of the United States. The western portion of the United States is serviced by DCMD West. 
DCMD West’s headquarters is located in Los Angeles California. DCMD International, with headquarters 


in Fort Belvoir, Virginia, is responsible for all contracts performed in overseas facilities. In addition to the 


10 


three districts, DCMC maintains 8] Contract Administration Offices (CAOs) located throughout the world. 
Approximately half of these offices are located in individual contractor plants. Formerly known as 
Defense Plant Representative Offices, these on-site DCMC offices are responsible for contracts at that 
particular contractor’s site. The remaining DCMC offices service multiple contracts in designated 
geographic areas. 

A typical DCMC office is organized into the following five business groups under a Commander and 
Deputy Commander: Small Business Office, Office of Counsel, Management Support Group, Operations 
Group, and Technical Assessment Group. The Small Business office provides support to small businesses 
within the DCMC’s area of responsibility and encourages small business participation in DOD acquisitions. 
The office of Counsel provides legal counsel primarily to the Commander, Team leaders, and ACOs. The 
Management Support Group is responsible for administrative and information management services. The 
main groups within the DCMC office that provide contract management services are the Operations Group 
and the Technical Assessment Group. 

The Operations Group utilizes teams of multi-functional specialists to provide a wide range of contract 
management services. The teams are responsible for contract management within a specified geographic 
area. Team members include Administrative Contracting Officers (ACOs), Contract Administrators, 
Contract Management Analysts, Industrial Specialists and Quality Assurance Specialists. Team leaders are 
chosen based upon their management ability and broad range of experience and can come from any of the 
functional areas listed above. Teams under the Operations Group are primarily responsible for post-award 
administration and surveillance. 

The Technical Assessment Group is responsible for more specialized contract management services. 
This Group is comprised of the Preaward Survey Manager, Government Property specialists, 
Transportation and Packaging specialists, Termination Contracting Officers (TCOs), Quality Assurance 
staff personnel, engineers, specialized safety specialists, and cost/price analysts. The Technical 
Assessment Group is responsible for performing Preaward Surveys, pricing reviews, technical assessments, 


contract administration on terminated contracts, Progress Payment reviews, specialized safety reviews, 


if 


Government Property Control reviews and industrial base assessments. In addition to these functions, the 
Technical Assessment Group provides support to DCMC’s Operation Group and field Quality Assurance 
Representatives. 

Within DCMC’s Technical Assessment Group, the Preaward Survey Manager (PASM) is responsible 
for the conduct of all Preaward Surveys performed at the Contact Administration Office (CAO). The 
breadth of Preaward Surveys can be quite extensive and can require participation from a wide range of 
functional specialists. As such, one of the main responsibilities of the PASM is to coordinate the actions of 
a multi-disciplinary Preaward Survey team, as well as coordinate and schedule Preaward Surveys at 
numerous contractor facilities. These actions must be accomplished efficiently to ensure that Preaward 
Surveys are completed by customer need dates. 

C. PREAWARD SURVEY PROCEDURES 

As discussed previously, the Procuring Contracting Officer (PCO) must ensure that a prospective 
contractor is capable of performing satisfactorily on a Government contract prior to making a contract 
award. When information is not available on hand to facilitate a responsibility determination, the PCO can 
request that DCMC perform a Preaward Survey on the prospective Government contractor. To start the 
process, the PCO will submit a Standard Form (SF) 1403, Preaward Survey of Prospective Contractor, to 
the DCMC office assigned responsibility for managing contracts within the geographic proximity of the 
prospective contractor. The Procuring Activity lists all pertinent data on the Preaward Survey request 
form, such as solicitation number, proposed contract value, name, location and business size of the 
prospective contractor, points of contact, need date, and items being procured. In addition to this critical 
background information, the Procuring Activity indicates the factors which should be evaluated during the 
Preaward Survey and any special instructions or concerns. A copy of the solicitation should be attached to 
the Preaward Survey request to facilitate the evaluation of the prospective contractor’s capability to 
perform on the proposed contract. 

The most frequently requested factors for evaluation are Technical capability, Production capability, 


Quality Assurance capability and Financial capability. PCOs request accounting system and specialized 
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reviews less frequently. Accounting system reviews are normally requested on proposed cost- 
reimbursement type contracts or fixed-price type contracts with special financial assistance provisions such 
as progress payments. Under these circumstances, a prospective contractor must have an adequate 
accounting system which is capable of tracking and segregating costs. The specialized reviews include an 
assessment of the prospective contractor’s Government Property Control procedures, Transportation and 
Packaging capability, security controls and safety programs. These specialized reviews are performed on 
an as needed basis. 

Depending upon the scope of the Preaward Survey, numerous personnel from muli-disciplines may be 
required to perform portions of the Preaward Survey. Preaward Surveys on highly complex and technical 
requirements are costly and time consuming. These type surveys may require participation by ACOs, 
Contract Administrators, Industrial Specialists, Quality Assurance Specialists, technical engineers, 
cost/price analysts and/or other functional specialists. In addition to DCMC participation, the PCO and 
Procuring Activity technical representatives may participate in the Survey. The Federal Acquisition 
Regulation encourages PCOs to tailor their Preaward Survey requests, where appropriate, to only those 
factors or areas that are needed to support the responsibility determination and procurement [Ref. 115:p. 9- 
4]. This helps ensure that limited resources are employed effectively and lowers total acquisition costs. 

In addition to performing formal Preaward Surveys at a contractor’s facility, DCMC can perform a 
“Desk” Survey to obtain information from on-hand files, other Government and/or commercially 
available sources. If there is sufficient information on-hand, DCMCs are encouraged to use this 
information to satisfy a Procuring Activity’s request for contractor capability information. This practice 
avoids a costly formal Preaward Survey and frees-up scarce resources to satisfy other customer 
requirements. Additionally, the use of “Desk” Surveys provides information more quickly to the PCO and 
may reduce the Procuring Activity’s Procurement Administrative Lead Time (PALT). 

In light of the expense of Preaward Surveys, the Federal Acquisition Regulation (FAR) recognizes that 
it is not cost effective to perform Preaward Surveys on proposed low dollar value contracts in support of 


non-critical requirements. Specifically, the FAR states that when the proposed contract will have a fixed 


price at/or below $100,000 or will involve the acquisition of commercial items, the PCO should not request 
a Preaward Survey unless circumstances justify its cost. [Ref. 115:P. 9-3]. Adherence to this policy helps 
ensure that DCMC resources are used effectively. 

Timely tum-around of Preaward Surveys is one of DCMC’s highest priorities. Per the FAR, DCMC 
should normally complete the Preaward Survey within seven working days after receipt. [Ref. 115:p. 9-4] 
Although seven days is not much time to gather data, analyze the data and draft a formal report, some 
PCOs state that a seven day tum-around 1s not always responsive to their needs. In some cases PCOs need 
this information sooner to meet critical acquisition schedules and requirements. Conversely, several 
PASMs have indicated that it takes approximately 15 days to complete a high quality, professional 
Preaward Survey and approximately 30 days if a secondary survey is required on subcontractors. 

Upon completion of the Preaward Survey, the PASM forwards the report to the PCO with an “Award” 
or “No Award” recommendation based upon the survey findings. If the prospective contractor had all 
factors rated positive with the exception of one negative factor, the contractor will be found to be non- 
responsible. It should be noted that the Preaward Survey contains a recommendation from DCMC and that 
it is not binding on the PCO. The PCO makes the final responsibility determination. As such, the PCO 
may override a Negative Survey or elect not to award to a contractor who received an “Award” 
recommendation. 

D. RESPONSIBILITY DETERMINATION 

Before making a contract award, the PCO must ensure that the prospective contractor meets the 
general standards of responsibility in FAR Subpart 9.104-1. To be eligible for award of a Government 
contract, a prospective contractor must have the necessary production equipment and facilities, 
organization, management, technical skills, and financial resources to complete the contract. Additionally, 
the prospective contractor must have a satisfactory performance, integrity and business ethics record. If the 
prospective contractor currently does not have a required capability, the contractor must demonstrate the 
ability to acquire the necessary capability. [Ref. 115:p. 9-1] For instance, a contractor may not have a 


required machine or certified manufacturing process. The lack of these capabilities does not necessarily 
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make that contractor non-responsible. The contractor could demonstrate responsibility by obtaining those 
capabilities via a subcontractor. Another common example surrounds a prospective contractor’s financial 
capability. Many small businesses do not have sufficient assets or working capital on hand to perform on 
larger sized Government contracts. Thus, those small businesses tend to get a disproportionately high rate 
of Negative Preaward Surveys due to inadequate financial capability. Even though their working capital is 
Judged to be unsatisfactory, these contractors could be deemed responsible if they furnish documentation 
of financial backing from a parent company or letters of credit from a financial institution. 

Preaward Surveys help facilitate responsibility determinations and are a key element in the source 
selection process. However, it should be noted that the Preaward Survey recommendation is non-binding 
on the PCO. Prior to making a final responsibility determination, the PCO weighs the Preaward Survey 
findings and all relevant facts surrounding the procurement. The contractor’s capabilities may have 
changed for the better or worse after issuance of the Preaward Survey. Additionally the company may 
have aggressively pursued corrective actions after issuance of a Negative Preaward Survey or the 
contractor may have furnished updated, favorable information about their capabilities. Receipt of this new 
information may make the contractor eligible for award of the contract. 

E. PREAWARD SURVEY POLICIES 

There are no regulations or Service policies which prevent the PCO from making an award in the face 
of a Negative Preaward Survey. The PCO is empowered to obligate funds on behalf of the Government 
and has a fiduciary responsibility to ensure that only capable, responsible and responsive contractors 
receive Government contracts. Per the Federal Acquisition Regulation, the PCO's signing of the contract 
constitutes a determination that the prospective contractor is responsible with respect to that contract [Ref. 
115:p. 9-3]. In addition to signing the contract, the PCO is required to document all information used to 
determine a contractor's responsibility. To satisfy this requirement, the PCO may use contractor capability 
information obtained via a formal or "Desk" Preaward Survey, internal Procuring Activity records, other 
Government agencies, various contractor suspension, debarment, ineligibility and alert lists, and/or 


commercially available sources. 


In addition to supporting a determination of responsibility in the contract file, some Procuring 
Activities may require management review of the PCO's decision under certain circumstances. For 
instance, some Procuring Activities require the PCO to obtain approval from the Activity's Chief of 
Contracting prior to overriding a Negative Preaward Survey or awarding a contract to a contractor with a 
history of poor performance. Additionally, these local regulations may require the PCO to obtain counsel 
from the Activity's legal staff and cognizant functional specialists prior to contract award. The purpose of 
these management and functional specialist reviews is to ensure all pertinent facts surrounding the 
proposed award have been considered and to ensure that the PCO's decision is legally sound and prudent. 
F. SMALL BUSINESS ADMINISTRATION CERTIFICATE OF COMPETENCY PROGRAM 

The Federal Acquisition Regulation imposes special rules and procedures whenever a PCO determines 
a small business is non-responsible for reasons other than ineligibility, debarment or suspension [Ref. 
115:p. 9-3]. Under these circumstances, the PCO does not possess final authority for making the 
responsibility determination. The procurement process must be stopped and all pertinent information about 
the proposed contract and supporting non-responsibility documentation must be forwarded to the cognizant 
SBA Regional Office. Upon receipt of this information, the SBA office notifies the prospective contractor 
that the Procuring Activity found them to be non-responsible. The SBA Office then offers the prospective 
contractor the opportunity to apply for a SBA Certificate of Competency (COC). The COC document is 
SBA's certification that the prospective contractor is responsible to perform the proposed contract. 

If the prospective contractor believes that they are capable of performing the contract and elects to 
pursue a COC, the SBA Office will perform a capability survey on the contractor. This capability survey is 
similar to the Preaward Survey performed by DCMC. It assesses the prospective contractor's capability, 
competency, capacity, credit and integrity for the purpose of performing on the proposed contract. It does 
not, however, assess whether the prospective contractor is currently debarred, suspended or otherwise 
ineligible to receive a Government contract. 

Upon completion of the SBA capability survey, the SBA Regional Office will notify the prospective 


contractor and the PCO of the results. If SBA plans on issuing a COC, the SBA Office will notify the 
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PCO of this intention and provide a brief statement of its rationale for the decision. If the PCO disagrees 
with the SBA's COC decision, the PCO has two opportunities to overturn SBA's COC. The first 
opportunity is via an appeal to the SBA Regional Office. If this appeal is not successful, the last 
opportunity to overturn SBA's COC decision is via an appeal to the SBA Central Office. The final decision 
with respect to the issuance or denial of a COC resides with SBA. By regulation, the SBA COC is 
conclusive with the prospective contractor's responsibility. Under these circumstances the PCO must 
award a contract to a small business possessing a SBA COC, even if the award is made in the face of a 
PCO non-responsibility determination. 
G. SUMMARY 

Government policy dictates that only capable and responsible contractors receive Government 
contracts. Prior to award, the PCO is responsible for ensuring that the prospective contractor has the 
capabilities to perform on the proposed contract. When sufficient information about a contractor's 
capabilities is not on-hand, the PCO can request that DCMC perform a Preaward Survey on the prospective 
contractor. These surveys can assess a wide range of contractor capabilities, such as production , technical, 
quality assurance and financial capabilities. Formal Preaward Surveys are labor intensive and costly, 
especially if are large number of capability factors need evaluation. Therefore, the Federal Acquisition 
Regulation encourages PCOs to tailor their requests, where appropriate, to only those factors or areas that 
are needed to support the responsibility determination and procurement. Additionally, PCOs should only 
request a Preaward Survey when its cost is justified. 

The following Chapter discusses the methodology used to collect data for this study and presents the 


resulting research data. 


IW. METHODOLOGY AND DATA PRESENTATION 


A. GENERAL 

This chapter outlines the methodology used to conduct the research and presents the resulting research 
data. Analysis and interpretation of these data are included in Chapter IV. 

The purpose of this research is to examine the Department of Defense’s experience with contracts 
awarded in the face of a negative Preaward Survey and assess the effectiveness of the Preaward Survey 
process as a source selection and risk management tool. To accomplish this goal, the thesis attempts to 
answer the primary and subsidiary research questions presented in Chapter I. 

Data for this study were collected in five phases. The first phase consisted of a review of various 
Government and literature publications to gather background information on Preaward Survey policies, 
procedures and processes. Specific areas reviewed include the use of Preaward Surveys during source 
selection and policies governing the award of contracts in the face of a Negative Preaward Survey. The 
majority of data collected during this phase was used as background information during the course of this 
study. As such, the results of this research are presented in Chapter II with the rest of the background 
information on the Preaward Surveys. 

The second phase of data collection consisted of a review of Defense Contract Management District 
East (DCMDE) Preaward Survey files. Review of DCMDE top level metrics reports generated historical 
data on the total number of Positive and Negative Preaward Surveys conducted within the three DCMC 
Districts between March 1995 and December 1996. These data are presented in Chapter III. This 
information yielded the answer to research question number 3: What are the relative percentages of 
Positive and Negative Preaward Survey recommendations? Additionally, these data were used to select a 


representative DCMC office to conduct more in-depth research and data collection. 
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The third and fourth phases of data collection consisted of interviews with various Contract 
Administration Offices (CAO) and Procuring Activity personnel. Data collected during these interviews 
were used to gain insight into the effectiveness of the Preaward Survey process and its usefulness as a 
source selection and risk management tool. Data collected during CAO and Procuring Activity interviews 
are grouped and presented separately to facilitate a comparison of responses between those responsible for 
post-award administration and those responsible for preaward actions. Interview responses were used to 
determine the primary causes of Negative Preaward Survey recommendations and their impact on contract 
performance. Additionally, the responses were used to determine the reasons why PCOs award a contract 
in the face of a Negative Preaward Survey recommendation and to assess the correlation of contractor 
performance under those circumstances with Negative Preaward Survey findings. These data will be used 
to answer research questions 3 through 7. Data generated during this phase of data collection are presented 
in Chapter III. 

The fifth and last phase of data collection consisted of a detailed review of DCMC Philadelphia’s 1995 
Preaward Survey files. Data generated during this phase were used to determine the actual frequency and 
causes of negative Preaward Survey recommendations at a representative DCMC office. Additionally, 
data obtained from DCMC Philadelphia’s Negative Preaward Survey files were used to conduct follow-up 
research with Procuring Activity and CAO personnel. Data collected from follow-up research were used to 
validate CAO and Procuring Activity interview responses to research questions 4, 5 and 6. 

As discussed above, data collected during the first phase is contained in Chapter II. Data generated 
from the second through fifth phases of data collection are contained in this chapter and are organized by 
data source. Data are presented in the following order: (1) DCMD historical Preaward Survey statistics, (2) 
DCMC Philadelphia Preaward Survey statistics, (3) CAO personnel interview data, and (4) Procuring 
Activity interview data. Presentation of data by source facilitates comparison of CAO personnel responses 
with Procuring Activity responses. Additionally, presentation of data by source enables one to 
compare/validate DOD personnel perceptions with actual data generated from DCMC Philadelphia’s 


Preaward Survey files and follow-up interviews. An analysis of these data is contained in Chapter IV. In 
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that Chapter, the data will be presented in a fashion which facilitates analysis of the specific research 
questions. 
B. DEFENSE CONTRACT MANAGEMENT DISTRICT PREAWARD SURVEY STATISTICS 

This section of chapter III presents data contained from top level DCMD Preaward Survey reporting 
metrics. The purpose of collecting these data is to determine the total numer of Preaward Surveys 
conducted by DCMC, the relative percentages of positive and negative Preaward Survey recommendations 
and the respective Preaward Survey workload at individual DCMCs. Additionally, this information was 
used to select a representative DCMC to facilitate more in-depth Preward Survey data collection and 
follow-up research. 

Tables 1 and 2 depict the total number of Preaward Surveys conducted by the three DCMC Districts in 
1995 and 1996 respectively. (Table 1 reflects data collected from March 1995 through December 1995. 
March 1995 was the first month the Defense Logistics Agency (DLA) converted their Preaward Survey 
reporting requirements to a top level metrics system. Therefore, this study will only present historical 
Preaward Survey data for the last 10 months of 1995). As presented in Tables 1 and 2, DCMC conducted 
over seven thousand one hundred Preaward Surveys between March 1995 and December 1996 with the 
largest majority of surveys performed within DCMC’s East District (DCMDE). Across all three Districts, 
the percentage of negative Preaward Survey recommendations in both 1995 and 1996 was 23 percent. 
During both reporting periods, the highest percentage of negative recommendations reported by a District 
was 26 percent and the lowest percentage was 17 percent. Ofnote, DCMDE had the highest percentage of 
negative recommendations in 1995 and was tied with DCMC’s International District (DCMDI) for the 
highest percentage of negative recommendations in 1996. The relative percentage of negative 
recommendations for DCMDE and DCMC District West (DCMDW) did not vary significantly between 
1995 and 1996. The greatest fluctuation in the relative percentage of negative recommendations occurred 
within DCMC’s International District (DCMDI). The variation in DCMDI’s statistics is attributed to the 


relatively low percentage of Preaward Surveys performed by DCMDI. Any variation in either the number 
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TABLE 1 


TOTAL PREAWARD SURVEY BY DEFENSE CONTRACT MANAGEMENT DISTRICT 
MARCH 1995 to DECEMBER 1995 


DISTRICT TOTAL PAS POS PAS % POS NEG PAS % NEG 
DCMDE Z01e 1,484 74% 935 26% 
DCMDW 1,482 1,162 80% 300 20% 


DCMDI 212 eA 83% 36 17% 
TOTAL 3,714 2,843 17% 871 23% 


(Source: DCMC Records] 


TABLE 2 


TOTAL PREAWARD SURVEY BY DEFENSE CONTRACT MANAGEMENT DISTRICT IN 1996 


DISTRICT TOTAL PAS POS PAS % POS NEG PAS % NEG 
DCMDE 1,818 1,339 14% 479 26% 
DCMDW 1,346 1115 83% 228 17% 


DCMDI 246 183 14% 63 26% 
TOTAL 3,410 2,640 17% 770 23% 


[Source: DCMC Records] 
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TABLE 3 


TOTAL PREAWARD SURVEY BY DEFENSE CONTRACT MANAGEMENT DISTRICT EAST 
MARCH 1995 to DECEMBER 1995 


DCMC 


ORLANDO 
RAYTHEON 
LONG ISLAND 
BIRMINGHAM 
GRUM ST AUG 
GRUM MELBO 
LOCKHEED MA 
MICHOUD-STEN 
CLEARWATER 
PRATT & WHITN 
BOSTON 
HARTFORD 
SYRACUSE 
HAMILTON STD 
GEN ELECTRIC 
GRUMM BETHP 
SIKORSKY 
INDIANAPOLIS 
GRAND RAPIDS 
CLEVELAND 
DAYTON 
DETROIT 

GENL DYNAMIC 
UNITED DEFEN 
NEW YORK 
SPRINGFIELD 
ALLIED SIGNAL 
STRATFORD 
BALTIMORE 
READING 
PITTSBURGH 
PHILADELPHIA 
WESTINGHOUS 
BOEING HELI 
PEMCO 
ATLANTA 
APMO GA 
CCMO WASH 
LORAL DEFEN 


TOTAL DCMDE 


(Source: DCMC Records] 


TOTAL PAS 


2,019 


POS PAS 


1,484 


23 


% POS 


67% 
100% 
69% 
70% 
N/A 
N/A 
100% 
N/A 
72% 
N/A 
86% 
63% 
94% 
N/A 
N/A 
N/A 
N/A 
79% 
57% 
68% 
80% 
78% 
N/A 
N/A 
72% 
73% 
N/A 
81% 
78% 
73% 
85% 
62% 
100% 
N/A 
N/A 
69% 
N/A 
N/A 
N/A 


74% 


NEG PAS 


35 


539 


% NEG 


33% 
0% 
31% 
30% 
N/A 
N/A 
0% 
N/A 
28% 
N/A 
14% 
37% 
6% 
N/A 
N/A 
N/A 
N/A 
21% 
43% 
32% 
20% 
22% 
N/A 
N/A 
28% 
27% 
N/A 
19% 
22% 
27% 
15% 
38% 
0% 
N/A 
N/A 
31% 
N/A 
N/A 
N/A 


26% 


TABLE 4 


TOTAL PREAWARD SURVEY BY DEFENSE CONTRACT MANAGEMENT DISTRICT WEST 


DCMC 


CHICAGO 

SAN ANTONIO 
VAN NUYS 
LOCKHEED MA 
PHOENIX MESA 
DALLAS 
PHOENIX 
E-SYSTEMS GR 
TEXAS INSTRU 
LORAL VOUGHT 
HUGHES TUSC 
STEWERT & ST 
SANTA ANA 
McDONNELL D 
SAN DIEGO 
SAN FRANCISC 
BOEING SEA 
SEATTLE 
HUGHES LA 
ROCKWELL C 
NORTHROP GR 
BOEING WITCHI 
DENVER 
THIOKOL 

BELL HELICO T 
McDONNELL D 
ST LOUIS 

TWIN CITIES 
WITCHITA 


TOTAL DCMDW 


(Source: DCMC Records] 


TOTAL PAS 


POS PAS 

125 95 
138 101 
175 1395 
2 2 
0 0 
165 124 
169 oe 
1 1 
Ss) fs) 
0 0 

9 9 
0 0 
149 120 
2 Z 
60 53 
130 104 
2 2 
13 54 
0 0 
0 0 
0 0 
0 0 
96 81 
= . 
0 0 
0 0 
84 49 
49 43 
S12) 36 
1,482 1,182 
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MARCH 1995 to DECEMBER 1995 


% POS 


76% 
73% 
77% 
100% 
N/A 
75% 
93% 
100% 
100% 
N/A 
100% 
N/A 
81% 
100% 
88% 
80% 
100% 
14% 
N/A 
N/A 
N/A 
N/A 
84% 
100% 
N/A 
N/A 
58% 
88% 
92% 


80% 


NEG PAS 


ooh 


W =a 
WOnoed0cn”creononcncddcdda & 


300 


% NEG 


24% 
27% 
23% 
0% 
N/A 
25% 
7% 
0% 
0% 
N/A 
0% 
N/A 
19% 
0% 
12% 
20% 
0% 
26% 
N/A 
N/A 
N/A 
N/A 
16% 
0% 
N/A 
N/A 
42% 
12% 
8% 


20% 


DCMC 


AUSTRALIA 
BRUSSELS 
ISRAEL 
KOREA 
PUERTO RICO 
ANKARA 
SAUDI LA 
KUALA LUMPUR 
SAUDI ARABIA 
AMERICAS 
NORTH EUR 
SOUTH EUR 
PACIFIC 


TOTAL DCMDI 


[Source: DCMC Records] 


TABLE 5 


TOTAL PREAWARD SURVEY 
DEFENSE CONTRACT MANAGEMENT DISTRICT INTERNATIONAL 
MARCH 1995 to DECEMBER 1995 


TOTAL PAS 


ZN3 


POS PAS 


25 


177 


% POS 


N/A 
88% 
80% 
67% 
86% 
90% 
50% 
75% 

N/A 
92% 
73% 
95% 
64% 


83% 


NEG PAS 


OoOwWwA NM CO HK ABH WNHAN | CO 


W 
(ep) 


% NEG 


N/A 
13% 
20% 
33% 
14% 
10% 
50% 
25% 

N/A 

8% 
27% 
5% 
36% 


17% 


TABLE 6 


TOTAL PREAWARD SURVEY 
BY DEFENSE CONTRACT MANAGEMENT DISTRICT EAST IN 1996 


DCMC TOTAL PAS 
ORLANDO 83 
RAYTHEON 4 
LONG ISLAND 109 
BIRMINGHAM eh. 
GRUM ST AUG 0 
GRUM MELBO 0 
LOCKHEED MA 11 
MICHOUD-STEN 0 
CLEARWATER 87 
PRATT & WHITN 0 
BOSTON 100 
HARTFORD 105 
SYRACUSE 31 
HAMILTON STD 0 
GEN ELECTRIC 0 
GRUMM BETHP 0 
SIKORSKY 0 
INDIANAPOLIS ZA) 
GRAND RAPIDS 18 
CLEVELAND 64 
DAYTON Ai i2 
DETROM Tes 
GENL DYNAMIC 0 
UNITED DEFEN 0 
NEW YORK 73 
SPRINGFIELD 96 
ALLIED SIGNAL 1 
STRATFORD 23 
BALTIMORE 294 
READING 47 
PITTSBURGH 56 
PHILADELPHIA 131 
WESTINGHOUS 2 
BOEING HELI 0 
PEMCO 0 
ATLANTA 176 
APMO GA 0 
CCMO WASH 0 
LORAL DEFEN 0 
TOTAL DCMDE 1,818 


[Source: DCMC Records] 


POS PAS 


139 


OOo © 


ipood 


FAS. 


% POS 


89% 
100% 
66% 
58% 
N/A 
N/A 
100% 
N/A 
64% 
N/A 
72% 
17% 
84% 
N/A 
N/A 
N/A 
N/A 
90% 
78% 
63% 
77% 
73% 
N/A 
N/A 
84% 
83% 
N/A 
70% 
71% 
74% 
79% 
66% 
100% 
N/A 
N/A 
79% 
N/A 
N/A 
N/A 


74% 


NEG PAS 


479 


% NEG 


11% 
0% 
34% 
42% 
N/A 
N/A 
0% 
N/A 
36% 
N/A 
28% 
23% 
16% 
N/A 
N/A 
N/A 
N/A 
10% 
22% 
38% 
23% 
27% 
N/A 
N/A 
16% 
17% 
N/A 
30% 
29% 
26% 
21% 
34% 
0% 
N/A 
N/A 
210 
N/A 
N/A 
N/A 


26% 


DCMC 


CHICAGO 

SAN ANTONIO 
VAN NUYS 
LOCKHEED MA 
PHOENIX MESA 
DALLAS 
PHOENIX 
E-SYSTEMS GR 
TEXAS INSTRU 
LORAL VOUGHT 
HUGHES TUSC 
STEWERT & ST 
SANTA ANA 
McDONNELL D 
SAN DIEGO 
SAN FRANCISC 
BOEING SEA 
SEATTLE 
HUGHES LA 
ROCKWELL C 
NORTHROP GR 
BOEING WITCHI 
DENVER 
THIOKOL 

BELL HELICO T 
ST LOUIS 

TWIN CITIES 
WITCHITA 


TOTAL DCMDW 


[Source: DCMC Records] 


TABLE 7 


TOTAL PREAWARD SURVEY 
BY DEFENSE CONTRACT MANAGEMENT DISTRICT WEST IN 1996 


TOTAL PAS 


170 
oi 
120 


POS PAS 


4 
1 
4 


ea 


27 


46 
14 
02 


18 


% POS 


86% 
83% 
85% 
N/A 
N/A 
81% 
85% 
N/A 
100% 
N/A 
N/A 
N/A 
80% 
50% 
94% 
76% 
100% 
83% 
N/A 
N/A 
N/A 
N/A 
89% 
100% 
N/A 
73% 
89% 
100% 


83% 


NEG PAS 


NO 


AN 


— 
OA NOOO WOOD CO FPO ONMNH WOeDODT0W CO WN 


— 


228 


% NEG 


14% 
17% 
15% 
N/A 
N/A 
19% 
15% 
N/A 
0% 
N/A 
N/A 
N/A 
20% 
50% 
6% 
24% 
0% 
17% 
N/A 
N/A 
N/A 
N/A 
11% 
0% 
N/A 
27% 
11% 
0% 


17% 


TABLE 8 


TOTAL PREAWARD SURVEY 
BY DEFENSE CONTRACT MANAGEMENT DISTRICT INTERNATIONAL IN 1996 


DCMC TOTAL PAS POS PAS % POS NEG PAS % NEG 
AUSTRALIA 1 1 100% 0 0% 
BRUSSELS 1 1 100% 0 0% 
ISRAEL 1 1 100% 0 0% 
KOREA 122 2 17% 10 83% 
PUERTO RICO 1 1 100% 0 0% 
ANKARA 0 0 N/A 0 N/A 
SAUDI LA 0 0 N/A 0 N/A 
KUALA LUMPUR 1 1 100% 0 0% 
AMERICAS Ss. 42 76% Ve 24% 
SAUD! ARABIA 1 1 100% 0 0% 
NORTH EUR 11 9 82% Z 18% 
SOUTH EUR 118 101 86% 17 14% 
PACIFIC 444 23 52% 21 48% 
TOTAL DCMDI 246 183 74% 63 26% 


(Source: DCMC Records] 
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duplication of a formal Preaward Survey with a “phone” survey. [Ref. 106] Given the above data and the 
large number of Preaward Surveys conducted, DCMC Philadelphia was judged to be a representative 
DCMC. 

C. DEFENSE CONTRACT MANAGEMENT COMMAND PREAWARD SURVEY DATA 

This section presents data obtained from DCMC Philadelphia’s 1995 Preaward Survey files and 
follow-up interviews with Procuring Activity and CAO personnel. The purpose of collecting these data is 
to determine the actual frequency and primary causes of negative Preaward Survey recommendations at a 
representative DCMC office. Additionally, these data were collected to determine the reasons why 
Procuring Contracting Officers (PCOs) award a contract in the face of a negative Preaward Survey 
recommendation, and to facilitate an analysis of contractor performance under those circumstances. 

As discussed above, DCMC Philadelphia was selected as a representative DCMC based upon their 
Preaward Survey Statistics and the large number of Preawards performed at that Command. It was 
determined that Preaward Survey files from 1995 would provide the best source of data to facilitate an 
examination of DOD’s experience with contracts awarded in the face of a Negative Preaward Survey 
recommendation. Selecting more current periods would yield sufficient data on the frequency of Negative 
Preaward Survey recommendations and their causes, but would not allow sufficient time to pass for the 
contractor to accumulate a performance history under a contract awarded in the face of a Negative 
Preaward Survey recommendation. Selecting files older than 1995 poses the risk that the associated 
contract files would be closed out and not readily accessible for review and analysis. Therefore, 1995 
Preaward Survey files were chosen to allow the contractor ample time to perform (or generate some 
performance history) and to increase the probability that associated contract files were still accessible. 

DCMC Philadelphia conducted a total of 202 Preaward Surveys during calendar year 1995. As part of 
this study, the following variables were recorded for each Preaward Survey conducted in 1995: 

1. Survey Serial Number 


2. Survey Type (Preaward or General Purpose) 


2g 


3. Preaward Survey Recommendation 

4. Whether the Survey was canceled prior to completion 

5. Whether the prospective contractor declined to participate in the Survey 

6. Dollar Value of the proposed contract 

7. Whether the requesting activity justified the need for a low dollar Survey (<$25K) 

8. Type contract (Fixed-Price or Cost-Reimbursement) 

9. Whether the proposed contract contained financing assistance (Progress Payments) 

10. Whether the prospective contractor was a large or small business 

11. The factors requested to be evaluated 

12. Results of each factor evaluated 

The following presents a discussion of the Preaward Survey factors reviewed. The Survey number was 
recorded to facilitate follow-up review and to ensure no records were duplicated. Survey Type refers to the 
type of survey requested. DCMC generally performs two types of Surveys, a Preaward Survey and a 
General Purpose Survey (sometimes referred to as a Capability Survey). The Preaward Survey is a detailed 
review of a prospective contractor’s capabilities to perform on a proposed contract. This type of survey 
makes a specific “award” or “no award” recommendation. General Purpose Surveys assess the 
contractor’s general capabilities and do not assess its capabilities to perform on a specific contract. It is 
important to separate the two types of surveys. General Purpose Surveys were annotated to exclude them 
from calculations of Preaward Survey recommendation percentages. Including General Purpose Surveys in 
the total base of surveys conducted would skew the data. Likewise, the number of Surveys canceled prior 
to completion were recorded to ensure an accurate calculation of recommendation percentages. Whether 
or not a contractor declined to participate in the Survey or subsequently withdrew their bid/offer was 
recorded to determine if there are any significant trends of indiscriminate bidding. 

The proposed dollar value of the contract and justification of low dollar Preaward Survey requests 
were recorded to determine if there were any unfavorable trends in those areas. The type contract and 


whether or not financial assistance provisions are contained in the proposed contract were recorded to 
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determine if they had any bearing on the Preaward Survey recommendation and subsequent contractor 
performance. 

The factors requested and results of each factor requested were recorded to determine the major causes 
of negative Preaward Survey recommendations. This information will also be used during follow-up 
research on contracts awarded in the face of a negative Preaward Survey recommendation. Additionally, 
this information will be used to determine if there is a correlation between a contractor’s performance and 
the findings contained in negative Preaward Survey recommendations. 

The following section presents the data collected during the Preaward Survey file review. Table 9 
provides a summary of surveys performed by DCMC Philadelphia in 1995. As indicated in this Table, 
DCMC Philadelphia conducted a total of 152 Preaward Surveys and 41 General Purpose Surveys during 
this period. Excluding General Purpose Surveys and canceled surveys, DCMC Philadelphia’s Negative 
Preaward Survey recommendation rate in 1995 was approximately 45 percent. 

Data concerning the dollar value of proposed contracts are contained in Table 10. This Table 
categorizes Preaward Surveys by proposed contract value. This Table presents these data at a total number 
of surveys conducted level, as well as at the Preaward Survey recommendation level. The average 
proposed contract value for all Preaward Surveys conducted in 1995 was approximately $9.3 million. In 
this particular case, the average proposed contract value is misleading. During 1995, DCMC Philadelphia 
conducted over 16 Preaward Surveys with proposed contracts exceeding $10 million. Six of these surveys 
were valued at over $50 million with the highest proposed value being in excess of $410 million. These 
large dollar value surveys skew the data and raise the overall average. As indicated in this Table, the 
median dollar value of proposed contracts is approximately $127 thousand, much lower than the average. 
These data reflect that the majority of Preaward Surveys conducted at DCMC Philadelphia are for 
proposed contract values between $25 thousand and $100 thousand. 

The bottom portion of Table 10 presents the proposed dollar value for all Preaward Surveys in which 
the prospective contractor declined to participate in the Survey. This category of data represents a subset 


of data contained in the Negative Preaward Survey category. As such, it is not added to either the 
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Negative category or Total category data. The data in this subset indicate that prospective contractors 
declined to participate in a total of 15 Preaward Surveys or approximately 10 percent of all surveys 
conducted in 1995. Additionally, these data indicate that 93 percent of these surveys had a proposed 
contract value of less than $100 thousand and that the median contract value was approximately $35 
thousand. It should be noted that declining to participate in a Preaward Survey results in an automatic 
Negative Preaward Survey recommendation. However, declining to participate in a Preaward Survey does 
not preclude a prospective contractor from receiving a contract award. The PCO may award a contract 
under these circumstances if sufficient information to make a responsibility determination is available from 
alternate sources. Given this insight, the data reflect that approximately 18 percent of the Negative 
Preaward Survey recommendations issued by DCMC Philadelphia in 1995 were the result of prospective 
contractors declining to participate in the Survey. 

The proposed contract value was also used to facilitate an analysis of low dollar value Preaward 
Survey requests. Per Subpart 9.1 of the Federal Acquisition Regulation, the PCO should not request a 
Preaward Survey unless circumstances Justify its cost. [Ref. 115:P. 9-3] Generally, surveys with proposed 
contract values below $25 thousand should not be requested unless the end item or service is for a critical 
application. Table 1] presents data on all Preaward Surveys requested which had a proposed contract 
value of less than $25 thousand. These data reflect that approximately 21 percent of all Preaward Surveys 
conducted at DCMC Philadelphia during 1995 had a proposed contract value of less than $25 thousand. Of 
these Survey requests, approximately 44 percent contained justifications for requesting a low dollar value 
Survey. 

Tables 12 and 13 contain data on the size of businesses surveyed and the proposed contract type 
contained in Survey requests. Business size, either large or small, is self reported by a prospective 
contractor on its offer or proposal. In turn, the PCO annotates this information on the Preaward Survey 
request. For this study, business size data were taken directly from the Preaward Survey request. These 
data reflect that approximately 75 percent of all Preaward Surveys were conducted on Small Businesses 


and that Small Businesses received approximately 84 percent of the Negative Preaward Survey 
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recommendations issued by DCMC Philadelphia in 1995. The data in Table 13 reflect approximately 95 
percent of all Preaward Surveys requests were for Fixed-Price type contracts. The high percentage of 
Fixed-Price type contracts is not surprising when viewed in context of the large number of Preaward 
Surveys performed on Small Businesses and the median dollar value of proposed contracts. 

In addition to collecting data on proposed contract types, this study collected data on the number of 
Preaward Survey requests with financial assistance arrangements. The most common form of financial 
assistance is Progress Payment billing. Progress Payment provisions are sometimes included in large dollar 
contracts with long periods of performance. PCOs may elect to include this provision in Fixed-Type 
contracts to assist with a contractor’s working capital and cash flow needs. Under the terms of this 
arrangement, the contractor can submit periodic invoices for payment prior to completion of the contract. 
Partial payment is then made based upon the contractor’s allowable incurred costs and progress to date. 
Contracts with Progress Payment provisions are subject to audit by the Defense Contract Audit Agency 
(DCAA) and the contractor must have an adequate accounting system to facilitate the segregation of costs. 
[Ref. 6} As indicated in Table 14, approximately 15 percent of all Preaward Surveys conducted in 1995 
contained some form of financial assistance provisions. These data reflect however, that the inclusion of 
financial assistance provisions had little or no influence on the outcome of Preaward Survey 
recommendations. These data reflect the percentage of Positive Survey and Negative Survey 
recommendations were approximately equal at 15.5 percent and 14.7 percent respectively. 

Table 15 depicts the factors requested by PCOs for Preaward Survey evaluation. Additionally, this 
Table presents the frequency of factor evaluation requests and the primary causes of Negative Preaward 
Survey recommendations. These data reflect that the most frequently requested factor is Production 
followed by Technical, Quality Assurance and Financial capability. As reflected in Table 15, these four 
factors also cause the majority of Negative Preaward Survey recommendations. The highest percentage of 
Negative Preaward Survey recommendations are caused by Production and Financial concerns. 

This section presents the data collected from Preaward Survey file follow-up interviews with CAO and 


Procuring Activity personnel. Data were obtained from Procuring Activity personnel to determine if any 
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TABLE 15 


FACTORS REQUESTED FOR PREAWARD SURVEY EVALUATION & EVALUATION RESULTS 


FACTORS 
TECHNICAL 
PRODUCTION 
QUALITY 
FINANCIAL 
ACCOUNTING 
PROPERTY 
TRANSPORTATION 
PACKAGING 
SECURITY 
SAFETY 
ENVIRONMENTAL 
OTHER 
WALSH HEALEY 
MEET DELIVERY 
PAST PERFORM 
ED! CAPABILITY 
HEALTH REGS 
OTHER COMMIT 
MIL-I-45208 STD 


TOTAL 


[Source: DCMC Philadelphia Records] 


(Reflects Data taken from 152 Preaward Survey Files) 


NUMBER OF 


FREQ 


REQUESTS REQUESTED 


85 


97 


85 


19 


14 


Z1 


Je 


55.9% 
63.8% 
55.9% 
52.0% 
9.2% 
3.3% 
1.3% 
13.8% 
2.6% 
9.9% 
0.0% 
13.8% 
3.3% 
1.3% 
0.7% 
1.3% 
RO 


0.7% 


100.0% 


39 


NR 
POS 


76 


73 


64 


95 


14 


= © f9 -] |] W 


347 


PCT 
POS 


89.4% 
15.3% 
Too u6 
69.6% 
100.0% 
100.0% 
100.0% 
95.2% 
100.0% 
100.0% 
N/A 
52.4% 
60.0% 
50.0% 
100.0% 
100.0% 
0.0% 


100.0% 


78.7% 


NR 
NEG 


9 


24 


21 


24 


ONOO- N 


94 


PCT 
NEG 


10.6% 
24.7% 
24.7% 
30.4% 
0.0% 
0.0% 
0.0% 
4.8% 
0.0% 
0.0% 
N/A 
47.6% 
40.0% 
50.0% 
0.0% 
0.0% 
100.0% 


0.0% 


215% 


PCT OF 


ALL PAS 


5.9% 
15.8% 
13.8% 
15.8% 

0.0% 

0.0% 

0.0% 

0.7% 

0.0% 

0.0% 

0.0% 

6.6% 

1.3% 

0.7% 

0.0% 

0.0% 

13% 


0.0% 


61.8% 


of the 68 Negative Preaward Surveys issued by DCMC Philadelphia in 1995 resulted in a contract award to 
a contractor who had received a Negative Preaward Survey. If the contract was awarded to another source, 
no further information was sought from Procuring Activity personnel. However, if the PCO or Buyer 
indicated that a contract was awarded in the face of a Negative Preaward Survey, data were obtained on the 
associated contract number and the reasons for overriding DCMC’s “No Award” recommendation. Data 
generated during these follow-up interviews are presented in Table 16. These data indicate that PCOs 
awarded a contract in the face of a Negative Preaward Survey recommendation approximately 13 percent 
of the time during 1995. The reasons cited by PCOs for making these awards is contained in Table 17. As 
reflected in this Table, recent satisfactory performance on similar contracts was cited as the most frequent 
reason for overriding a “No Award” recommendation. It should be noted that all but two of the contract 
awards issued in the face of a Negative Preaward Survey were initiated by the PCOs responsible for those 
procurements. The two contracts awarded under the Small Business Administration’s Certificate of 
Competency (COC) Program would not have been awarded by the respective PCOs had they had complete 
control over the acquisition. In these two cases, the PCOs concurred fully with the findings contained in 
the Negative Preaward Surveys. 

The remainder of this section presents data collected from follow-up interviews with DCMC 
Philadelphia personnel. These data were collected to determine contractor performance on the overridden 
contracts identified above and to determine if contract performance failures line up with associated 
Negative Preaward Survey findings. Table 18 presents DCMC Philadelphia’s overall assessment of 
contractor performance under the nine contracts awarded in the face of a Negative Preaward Survey. 
Although subjective in nature, these assessments were based upon current or recent historical contractor 
performance on the specific contracts in question. These data indicate that 67 percent of the overridden 
contracts have a performance rating of satisfactory or excellent, while the remainder were rated either 
marginal or unsatisfactory. Specific performance deficiencies noted on the overridden contracts are listed 
in Table 19. As depicted, the most frequent problems noted on the overridden contracts were late 


deliveries and quality assurance problems. 
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TABLE 16 


CONTRACTS AWARDED IN THE FACE OF A NEGATIVE PREAWARD SURVEY 


DESCRIPTION NUMBER Pet 
Total Preaward Surveys Conducted in 1995 152 

Total Positive PAS Recommendations 84 55.3% 
Total Negative PAS Recommendations 68 44.7% 
Contracts Awarded in the Face of a Negative PAS 9 13.2% 


[Source: DCMC Philadelphia Records & Procuring Activity Interviews] 


TABLE 17 


PCO REASONS 
FOR AWARDING A CONTRACT IN THE FACE OF A NEGATIVE PREAWARD SURVEY 


FREQ PCT OF 
REASON CITED TOTAL 
Obtained Financial Data from Dunn & Bradstreet 1 RTS 
Recent Satisfactory Performance on Similar Contract(s) 3 PAS RS bi 
SBA Issued Certificate of Competency (COC) Za 15.4% 
Contractor Presented Letter of Credit and/or Pledges 2 15.4% 
of Financial Backing 
Procuring Activity Deleted or Reduced QA and/or 2 15.4% 
Higher Inspection Requirement(s) 
Urgent Requirement in Support of National Defense 1 Je 
No Other Source or It Would Take Too Long to Qualify 1 7.7% 
an Alternate Source 
Contractor Corrected Deficiency Noted in NEG PAS 1 7.7% 
TOTAL 13 


[Source: Procuring Activity Interviews] 


TABLE 18 


CONTRACTOR PERFORMANCE UNDER CONTRACTS 
AWARDED IN THE FACE OF A NEGATIVE PREAWARD SURVEY 


OVERALL PERFORMANCE ASSESSMENT NUMBER Poa 

Excellent 3 33.3% 
Satisfactory 3 332576 
Marginal 2 22.2% 


Unsatisfactory 1 11.1% 


(Source: DCMC Philadelphia Personnel Interviews] 


TABLE 19 


CONTRACTOR PERFORMANCE PROBLEMS NOTED UNDER CONTRACTS 
AWARDED IN THE FACE OF A NEGATIVE PREAWARD SURVEY 


SPECIFIC PERFORMANCE ASSESSMENT NUMBER Tae 
Contract Terminated for Default 0 N/A 
Contract Terminated for Convenience 1 25.0% 
Late Deliveries 2 50.0% 
Production Problems 0 N/A 
Non-Conforming Products 0 N/A 
Quality Problems 1 25.0% 
Required Excessive Government Assistance/Oversight 0 N/A 


TOTAL 4 


[Source: DCMC Philadelphia Personnel Interviews] 
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Table 20 provides a comparison of actual contract performance on overridden contracts with 
associated findings contained in Negative Preaward Surveys. In the case of the nine overridden contracts, 
these data indicate that there is not a very high correlation between contractor performance and Negative 
Preaward Survey findings. These data are not surprising when viewed together with the specific reasons 
PCOs gave for overriding the “No Award” recommendations. As depicted in Table 17, the majority of 
reasons PCOs cited for contract award seemed justified under the circumstances and appeared to reduce the 
risk of contract performance problems and/or default. Although not evident in Table 17, there appeared to 
be reduced risk even with the contract award based upon the “No Other Source” and “Urgent 
Requirement” reasons. These statements, plus a third justification, pertain to one contract. In this particular 
case, the contractor had received a Negative Preaward Survey due to insufficient working capital and 
negative net worth. However, prior to making a final responsibility determination, the PCO had received 
formal pledges of financial backing from the prospective contractor’s parent company. The contractor was 
subsequently found responsible after receipt of financial backing. This new financial information, along 
with the sole source and compelling National urgency reasons, were used as justification for overriding 
DCMC Philadelphia’s “No Award” recommendation. 

D. CONTRACT ADMINISTRATION OFFICE PERSONNEL INTERVIEWS 

During this phase of data collection, interview questions were given to personnel at DCMC 
Philadelphia and 14 other DCMCs located throughout the Continental United States. The other DCMCs 
were selected based upon the size of their Command and the relatively large number of Preaward Surveys 
performed by those offices in 1995 and 1996. Personnel were selected to participate in this study based 
upon their likely involvement in the Preaward Survey process and post-award contract administration. A 
total of 26 responses were received from DCMC Philadelphia. Each of the other 14 DCMCs contacted 
provided one response each for a total of 40 responses received. Data collected from these interviews were 
used to gain insight into the effectiveness of the Preaward Survey process and its usefulness as a source 
selection and risk management tool. Specific interview questions used during this portion of the study are 


contained in Appendix A. 


TABLE 20 


COMPARISON OF NEGATIVE PREAWARD SURVEY FINDINGS 
WITH CONTRACTOR PERFORMANCE UNDER CONTRACTS AWARDED 
IN THE FACE OF A NEGATIVE PREAWARD SURVEY 


NEGATIVE ACTUAL 

EVALUATION FACTOR PAS FINDINGS PERFORMANCE CORRELATION 
Technical Capability 0 N/A N/A 
Production Capability Zz 1 50.0% 
Quality Assurance Capability 3 2 66.7% 
Financial Capability 5 0 0.0% 
Walslh Healey 1 0 0.0% 
Contractor Non-Responsive 1 0 0.0% 
TOTAL 12 3 25.0% 


(Source: DCMC Philadelphia Files and DCMC Philadelphia Personnel Interviews] 
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The majority of the questions ask the respondent to give the frequency of a given event or process. The 
answers to these type questions are usually given as a percentage. In some cases, the questions ask the 
respondent to rank a particular event or process. The scales for these type questions are annotated on 
associated Tables contained in this section. 

In general, the questions were designed to solicit answers and perceptions based upon the respondent’s 
personnel experience. Where appropriate, respondents were asked to mark the answer “Not Applicable” or 
‘Don’t Know” if they had no personnel experience or knowledge of a particular subject or event. In order 
not to skew the data, all “Not Applicable,” “Don’t Know,” and omitted responses were excluded from 
question summary calculations. 

The following provides a summary of the responses to CAO interview questions. The position 
description and years experience of those who responded to the interview questions are contained in Tables 
21 and 22, respectively. The majority of responses were received from Industrial Specialists, who are 
responsible for assessing and monitoring contractors’ technical and production operations. In addition to 
the above responsibilities, Industrial Specialists help perform Preaward Surveys. Of note, over 87 percent 
of respondents have over 10 years of Government acquisition experience. None of the respondents had 
less than 2 years of experience. 

One concern voiced by CAO personnel is that they get little feedback on Preaward Surveys from 
Procuring Activities. Often time they do not know that a contract was awarded in the face of a negative 
Preaward Survey until after the contractor experiences problems. [Ref. 106] To quantify the magnitude of 
this situation, CAO personnel were asked to list the percentage of time they or their command were 
notified of a contract awarded in the face of a Negative Preaward Survey. The responses to this question 
are contained in Table 23. These data indicate that CAO personnel are notified of overridden Negative 
Preaward Surveys approximately 7 percent of the time. Of note, 69 percent of respondents stated that they 
never receive notice of such contract awards. 

CAO personnel were then asked to list the frequency with which PCOs award a contract in the face of 


Negative Preaward Survey. The responses to this question are depicted in Table 24. The weighted average 
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TABLE 21 


CONTRACT ADMINISTRATION OFFICE INTERVIEWS 
BY POSITION DESCRIPTION 


POSITION NUMBER 


Administrative Contracting Officer (ACO) 
Contract Administrator 

Industrial Specialist 

Quality Assurance Specialist 

Cost/Price Analyst 

Preaward Survey Manager (PASM) 
Contracts Management Team Leader 
Specialized Safety Manager 


as 
]—“"NOONMNWwWO A 


i 
© 


Total Number of CAO Interviews 


(Source: DCMC Interviews] 


TABLE 22 


EXPERIENCE LEVEL 
OF CONTRACT ADMINISTRATION OFFICE PERSONNEL 


YEARS EXPERIENCE NUMBER 


Less Than 2 Years 
2 to 5 Years 

5 to 10 Years 
Over 10 Years S35 


W NM © 


Total Number of CAO Interviews 40 


(Source: DCMC Interviews] 
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TABLE 23 


PROCURING CONTRACTING OFFICER NOTIFICATION 
OF OVERRIDDEN NEGATIVE PREAWARD SURVEYS 


FREQUENCY NUMBER 


No 
~J 


0 Percent 
10 Percent 
20 Percent 
30 Percent 
40 Percent 
50 Percent 
60 Percent 
70 Percent 
80 Percent 
90 Percent 
100 Percent 
Don't Know or Not Applicable 


oOo oo - Oo - OF AHA NN 


Weighted Average 6.7% 


[Source: DCMC Interviews] 


TABLE 24 


FREQUENCY PROCURING CONTRACTING OFFICERS 
OVERRIDE NEGATIVE PREAWARD SURVEYS 


FREQUENCY NUMBER 


O Percent 
10 Percent 
20 Percent 
30 Percent 
40 Percent 
50 Percent 
60 Percent 
70 Percent 
80 Percent 
90 Percent 
100 Percent 
Don't Know or Not Applicable 


hBONMjS ORM WN] OA 


om 


Weighted Average 31.9% 


(Source: DCMC Interviews] 
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260 
6.666667 


830 
31.92308 


of all responses 1s approximately 32 percent, however the range of responses varies from zero to 90 
percent. The diversity of responses to this question may be attributed to the diversity of positions held by 
respondents. Some positions lend themselves to firsthand knowledge of this subject whereas other 
positions may not afford this experience. Preaward Survey Mangers (PASMs), by virtue of their 
responsibilities, may have somewhat greater insight into the number of overridden “No Award” 
recommendations than a Contract Administrator or Price/Cost Analyst. Of note, 35 percent of those 
interviewed had no knowledge or experience from which to answer this question. This comes as no 
surprise when viewed in the context of the low notification rate of overridden Negative Preaward Surveys 
contained in Table 23. 

The next question asked CAO personnel to provide the percentage of time PCOs are justified in 
awarding a contract to a contractor who had received a Negative Preaward Survey. As depicted in Table 
25, the weighted average of all respondents indicates that CAO personnel feel that 38.5 percent of these 
awards are properly justified by the PCO. As with the previous two questions, approximately 33 percent of 
interviewees had no basis or experience from which to answer this question. 

The next three questions ask CAO personnel to provide the top three reasons PCOs give for overriding 
Negative Preaward Survey recommendations and the percent of contracts completed satisfactory under 
those circumstances. For the purpose of these questions, satisfactory contract performance was defined as 
no or minimal problems noted during contract performance. The first, second and third most frequent 
reasons for awarding a contract in the face of a Negative Preaward Survey are presented in Tables 26, 27, 
and 28, respectively. As indicated in Table 26, CAO personnel believe that the issuance of a SBA 
Certificate of Competency is the most frequent reason cited by PCOs and that less than a third of these 
contracts are completed satisfactorily. The data in Tables 26, 27 and 28 indicate that ““No Other Source” 
and the “Contractor performed satisfactorily on recent contracts” are the second and third most frequent 
reasons cited for overriding a Negative Preaward Survey. 

To gain insight to the causes of Negative Preaward Surveys, CAO personnel were asked to rank the 5 


most common factors causing a Negative recommendation. The responses to this question are summarized 
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TABLE 25 


PERCENT OF PROCURING CONTRACTING OFFICER 
OVERRIDES JUSTIFIED 


FREQUENCY NUMBER 


O Percent 
10 Percent 
20 Percent 
30 Percent 
40 Percent 
50 Percent 
60 Percent 
70 Percent 
80 Percent 
90 Percent 
100 Percent 
Don't Know or Not Applicable 


WrAarhnM he Oh ANON 


—. 


Weighted Average 36.5% 


[Source: DCMC Interviews] 
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TABLE 26 


MOST FREQUENT REASON PROCURING CONTRACTING OFFICER 
OVERRIDES NEGATIVE PREAWARD SURVEY & PERCENT OF 
CONTRACTS COMPLETED SATISFACTORILY UNDER THOSE CIRCUMSTANCES 


PCT COMPLETES 
REASON NUMBER SATISFACTORILY 
No other source/sole source contractor i 47.1% 
Contractor performed satisfactorily on recent contracts 1 40.0% 
Financial condition mitigated with Progress Payments 0 N/A 
Meeting delivery schedule deemed not critical 0 N/A 
PCO willing to accept reduced quality due to urgent need 0 N/A 
SBA issued Certificate of Competency 16 32-536 
PCO received updated information since PAS issued 0 N/A 
Contractor successfully rebutted NEG PAS findings 0 N/A 
Reported deficiencies won't impact proposed contract 0 N/A 
Maintain industrial base 1 10.0% 
Best source of available contractor pool O N/A 
Defects/Deficiencies are deemed minor 0 N/A 
Weighted Average All Reasons 37.5% 


[Source: DCMC Interviews] 


TABLE 27 


SECOND MOST FREQUENT REASON PROCURING CONTRACTING OFFICER 
OVERRIDES NEGATIVE PREAWARD SURVEY & PERCENT OF 
CONTRACTS COMPLETED SATISFACTORILY UNDER THOSE CIRCUMSTANCES 


PCT COMPERETED 
REASON NUMBER SATISFACTORILY 
No other source/sole source contractor 8 25.0% 
Contractor performed satisfactorily on recent contracts 5 40.0% 
Financial condition mitigated with Progress Payments 1 80.0% 
Meeting delivery schedule deemed not critical 2 20.0% 
PCO willing to accept reduced quality due to urgent need 2 30.0% 
SBA issued Certificate of Competency 4 26.6% 
PCO received updated information since PAS issued 0 N/A 
Contractor successfully rebutted NEG PAS findings 0 N/A 
Reported deficiencies won't impact proposed contract 4 50.0% 
Maintain industrial base 0 N/A 
Best source of available contractor pool 1 30.0% 
Defects/Deficiencies are deemed minor 0 N/A 
Weighted Average All Reasons 30.5% 


(Source: DCMC Interviews] 
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TABLE 28 


THIRD MOST FREQUENT REASON PROCURING CONTRACTING OFFICER 
OVERRIDES NEGATIVE PREAWARD SURVEY & PERCENT OF 
CONTRACTS COMPLETED SATISFACTORILY UNDER THOSE CIRCUMSTANCES 


PCT COMPLETED 
REASON NUMBER SATISFACTORILY 
No other source/sole source contractor 2 20.0% 
Contractor performed satisfactorily on recent contracts 4 32.5% 
Financial condition mitigated with Progress Payments 3 40.0% 
Meeting delivery schedule deemed not critical 1 70.0% 
PCO willing to accept reduced quality due to urgent need 0 N/A 
SBA issued Certificate of Competency 1 20.0% 
PCO received updated information since PAS issued 2 UNKNOWN 
Contractor successfully rebutted NEG PAS findings S' 16.7% 
Reported deficiencies won't impact proposed contract 5 73.0% 
Maintain industrial base 0 N/A 
Best source of available contractor pool 0 N/A 
Defects/Deficiencies are deemed minor 1 50.0% 
Weighted Average All Reasons 42.3% 


[Source: DCMC Interviews] 
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in Table 29. A rank of | indicates that the respondent felt that this factor was the most common reason for 
a Negative recommendation, while a rank of 5 indicated that the respondent felt this was the fifth most 
common reason. Values were then assigned to each ranking to determine a weighted score and overall 
rank for each factor. Number | Rankings received the highest value of 5, while Number 5 Rankings 
received the lowest value of 1 (X,=5, X,=4, X,=3, X,=2 and X,=1). As depicted in this Table, production 
was Cited as the most common reason contractors receive a Negative Preaward Survey. Of note, non- 
responsiveness of the prospective contractor is the fifth most common reason. A contractor is deemed 
non-responsive when it fails to participate in the Preaward Survey or fails to provide requested 
documentation. 

The next interview question attempts to determine the correlation of Preaward Survey findings with a 
contractor’s performance under a contract awarded in the face of a Negative Preaward Survey. The 
responses to this question are contained in Table 30. These data indicate that CAO personeel feel that the 
contractor’s failures or deficiencies line up with associated Negative Preaward Survey findings over 80 
percent of the time. 

Negative Preaward Surveys contain detailed information about a prospective contractor’s capabilities 
and deficiencies. The next two questions concern the usefulness and use of Negative Preaward Survey 
information during post-contract administration. CAO personnel were asked to rank on a scale of zero to 
10 the usefulness of Negative Preaward Surveys during post-contract administration. A rank of zero 
indicates that the survey was of no use, while a rank of 10 indicates the survey was very useful. A rank of 
5 indicates the survey was of moderate use. The responses to this question are contained in Table 31. As 
indicated in this Table, the weighted average rank for this question was 6.78. This indicates that Negative 
Preaward Survey information is useful to those CAO personnel who responded to this question. Of note, 
over 48 percent of those who responded to this question gave this area the highest rank, signifying that they 
found this information to be very useful. When asked how they used this information during post-contract 


administration, CAO personnel provided the following comments and examples: 


52 


TABLE 29 


MOST COMMON REASONS CONTRACTORS 
RECEIVE NEGATIVE PREAWARD SURVEY 
(Scale: 1=Most Common, 5=Fifth Most Common Reason) 


RAW SCORING WEIGHTED OVERALL 
REASON #1 #2 #3 #4 #5 SCORE RANK 
Production is 3 5 0 1 93 1 
Financial 6 6 & 1 2 70 2 
Quality Assurance 2 8 2 S 2 56 3 
Technical 1 4 5 4 3 47 4 
Non-Responsive 1 2 0 5 3 26 5 
Accounting System 1 1 1 4 5 Zs 6 
Packaging 0 0 1 0 1 4 7 


Govt Property Control 0 0 0 1 1 3 8 


[Source: DCMC Interviews] 
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TABLE 30 


CORRELATION OF CONTRACTOR PERFORMANCE 
WITH NEGATIVE PREAWARD SURVEY FINDINGS 


FREQUENCY NUMBER 


O Percent 
10 Percent 
20 Percent 
30 Percent 
40 Percent 
50 Percent 
60 Percent 
70 Percent 
80 Percent 
90 Percent 
100 Percent 
Don't Know or Not Applicable 


onon- &- Oo 000 0 


— 


Weighted Average 80.5% 


[Source: DCMC Interviews] 


TABLE 31 


USEFULNESS OF NEGATIVE PREAWARD SURVEYS 
DURING POST-CONTRACT AWARD ADMINISTRATION 


RANK NUMBER 
0 No Use 2 
1 0 
2 3 
3 1 
4 0 
5 Moderately Useful 1 
6 2 
Yi S 
8 6 
9 2 
10 Very Useful 7 

Don't Know or Not Applicable (Pe 


Weighted Average 6.78 
[Source: DCMC Interviews] 
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10. 


tel. 


PA 


Performance history used to identify potential problem areas. 
Signifies that a Post-Award Orientation Conference may be warranted for this particular contract. 
Provides specific points of discussion for Post-Award Orientation Conferences. 


Information may cause the ACO to deny the use of Progress Payments or trigger pre-payment 
audits of Progress Payment requests. 


Helps determine the level of post-award surveillance and Government assistance. 


Used to remind contractors of deficiencies and to notify them that they need to implement 
appropriate corrective action plan(s). 


Used as a reference point from which to measure corrective actions implemented by the 
contractor. 


Used to establish controls and surveillance plans at subtier contractors and vendors. 
Helps in the identification of critical processes and surveillance review points. 


If multiple surveys are issued on the same contractor, the survey information is used to determine 
contractor performance trends. 


Unfavorable financial status triggers increased surveillance of the contractor’s performance and 
ongoing financial condition to ensure the contractor is not experiencing a loss on the contract. 


Information may cause the initiation of price realism analysis, especially if the contractor’s bid 
was significantly lower than the next highest bid. 


. If the contractor’s financial condition is marginal or unsatisfactory, this information may be used 


to request monthly cash flow statements to guard against potential performance problems. 


The last group of questions asked CAO personnel to provide recommended changes to the Preaward 
Survey process to enhance its use during source selection and post-award contract administration. The 
responses to these questions follow: 


ee) 


To increase the accountability of the SBA Certificate of Competency (COC) process, require SBA 
to conduct all post-contract award administration on SBA COC awarded contracts. 

Ensure PCOs do not award contracts to small businesses merely on the “threat” or expectation that 
the contractor will be issued a COC. 


When issuance of a COC is likely, require SBA to discuss its rationale with DCMC prior to 
issuance of the COC. 


Require SBA to furnish COC findings and its rationale for overriding DCMC’s “No Award” 
recommendation. This information should be sent to the appropriate Procuring Activity and 


cognizant CAO. 


With respect to SBA COCs, final responsibility determination should reside at the Procuring 
Activity vice at SBA. PCOs should have the authority to override a COC. 


»» 


Inte 


14. 


jis) 


16. 


SBA’s COC Program and processes should be evaluated. As appropriate, SBA’s COC program 
should be modified to ensure only responsible contractors receive contract awards. 


Disestablish the SBA’s COC Program in its entirety. This program duplicates efforts performed 
by DCMC. The SBA COC Program costs taxpayers an enormous amount of money to perform 
this mission with results that often result in additional outlays of time, money and resources. In 
this era of limited resources and budget constraints, the Government can ill afford the cost of 
duplicative programs. 


Require PCOs to notify the cognizant CAO office anytime they award a contract in the face of a 
Negative Preaward Survey. Ensure wide distribution of this information throughout the CAO 
OfLice: 


Whenever a Preaward Survey is conducted, ensure that the Preaward Survey Serial Number is 
included on the face of the contract, regardless of whether the recommendation was positive or 
negative. 


. Widen the internal distribution of Negative Preaward Surveys within the CAO office. This 


information can be potentially useful to ACOs, Contract Administrators, Industrial Specialists, 
Quality Assurance Specialists, Cost/Price Analysts, specialized functional specialists, Team 
Leaders and DCMC management. 


Increase the use and frequency of Post-Award Orientation Conferences whenever a contract is 
awarded in the face of a Negative Preaward Survey. During this Conference, ensure the 
contractor fully understands contract requirements and understands the requirement to correct 
deficiencies noted in the Negative Preaward Survey. Ensure appropriate CAO team members 
participate in this conference and post-award planning. 


. Include a “Trend Analysis” section in the Preaward Survey and require this information whenever 


a prospective contractor has received multiple Preaward Surveys within a 2 to 3 year period. 


. Make it easier to terminate a contract for default whenever a contractor experiences self induced 


problems on a contract awarded in the face of a “No Award” recommendation. 


Establish more specific guidelines for the functional specialists who perform Preaward Surveys. 
Current guidance does not delineate working guidelines for functional specialists. Establish a 
Preaward Survey Procedures Desk Guide and better training for functional specialists. These 
guidelines should make the Preaward Survey Process more uniform across the various DCMCs. 
This should give Procuring Activities more confidence in DCMC’s ability to evaluate contractors’ 
capabilities. 


Whenever a contract is awarded in the face of a Negative Preaward Survey, require the contractor 
to post a Performance Bond to assure satisfactory performance. 


Add a third recommendation block of “Award Not Recommended” to the Preaward Survey form. 
The performance of some contractors are marginal, but not clear enough to distinguish between an 
“Award” or “No Award” recommendation. This third category could be used to signify that 
excessive Government oversight or assistance beyond what is contractually required may be 
required to ensure contract completion. This category would not completely eliminate a 
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contractor from the source selection, but would make it easier to make an award based upon “Best 
Value” to the Government. 


17. Preaward Surveys are under utilized. Too many marginal companies are awarded contracts 
without benefit of determining their capabilities via a Preaward Survey. Ensure PCOs and Buyers 
request a Preaward Survey whenever a clear cut responsibility determination can not be made. 


18. Advise contractors that is mandatory to correct problems noted in Preaward Surveys. 


19. Ensure CAO Team members, including the ACO, are invited to participate in Preaward Surveys 
on “high risk” contractors. 


20. Revise the current contract administration data base to capture and report contractor performance 
data under contracts awarded in the face of a Negative Preaward Survey. 


21. Assign a higher value and weighting to Preaward Survey findings during source selections. 


22. Preaward Surveys should be recognized as part of DCMC’s “Early Contract Administration 
Services (CAS) program. One part of this program focuses on the evaluation of contractors’ 
performance capabilities during source selection. Early CAS helps PCOs select capable 
contractors and identify potential performance risks. The Preaward Survey process should be 
made an integral part of Early CAS. 


23. PCOs should allow DCMC more time to conduct Preaward Surveys. The FAR states that 
Procuring Activities should allow at least seven days to perform a Preaward Survey. In reality, it 
takes approximately 15 days to perform a thorough survey and about 30 days if a secondary 
survey on a subcontractor 1s required. 


24. Ensure PCOs and Buyers understand the differences between a General Purpose 
Survey/Capability Survey and a Preaward Survey. A number of PCOs are using General Purpose 
Surveys for responsibility determinations. This type of survey does not evaluate a contractor’s 
capability to perform on a specific contract. Contract terms and requirements change with each 
procurement. Likewise, the contractor’s management, financial situation, shop loading and 
production capabilities change over time. Only a Preaward Survey assesses a contractor’s current 
capabilities to perform on a specific contract. 

25. Require CAQOs to establish a specific surveillance plan for each contractor who has received a 
contract in the face of a Negative Preaward Survey. The plan should address actions to mitigate 
the risks associated with Negative Preaward Survey findings. 

E. PROCURING ACTIVITY PERSONNEL INTERVIEWS 

During this phase of data collection, interview questions were given to Procuring Activity Personnel at 
Navy Inventory Control Point (NAVICP) Philadelphia, NAVICP Mechanicsburg, Defense Industrial 
Supply Center (DISC) Philadelphia, Fleet Industrial Supply Center (FISC) Norfolk Detachment 


Philadelphia, U.S. Army Aviation and Troop Command and Warner Robins Air Logistics Center. 


Personnel were selected to participate in this study based upon their involvement in the acquisition process 
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and likelihood of participating in the Preaward Survey process. A total of 52 responses were received. 
The interview questions are similar to the questions given to CAO personnel. However, data collected 
from these interviews were used to gain insight into the effectiveness of the Preaward Survey process from 
a Procuring Activity’s perspective. Specific interview questions used during this portion of the study are 
contained in Appendix B. 

As was the case with CAO interview questions, the majority of the questions ask the respondent to 
give the frequency of a given event or process. The answers to these type questions are usually given as a 
percentage. In some cases, the questions ask the respondent to rank a particular event or process. The 
scales for these type questions are annotated on associated Tables contained in this section. 

In general, the questions were designed to solicit answers and perceptions based upon the respondent’s 
personnel experience. Where appropriate, respondents were asked to mark the answer “Not Applicable” or 
“Don’t Know” if they had no personnel experience or knowledge of a particular subject or event. In order 
not to skew the data, all “Not Applicable,” “Don’t Know” and omitted responses were excluded from 
question summary calculations. 

The following provides a summary of the responses to Procurement Activity interview questions. The 
position description and years experience of those who responded to the interview questions are contained 
in Tables 32 and 33, respectively. The majority of responses were received from PCOs and Contract 
Specialists. Of note, over 78 percent of respondents have over 10 years of Government acquisition 
experience. 

Procuring Contracting Officers request Preaward Surveys to gain insight into a prospective 
contractor’s capability to perform on a specific contract. To determine the value of this product, Procuring 
Activity personnel were asked to rate the usefulness of information contained in Preaward Surveys during 
the source selection process. Responses to this question are contained in Table 34. A rank of zero 
indicates that the Surveys are of no use, while a rank of 10 indicates the Surveys are a very useful tool. As 


reflected in Table 34, the weighted average rank given to this question was 7.58. These data indicate that 
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TABLE 32 


PROCUREMENT ACTIVITY INTERVIEWS 
BY POSITION DESCRIPTION 


POSITION NUMBER 


Procuring Contracting Officer (PCO) 24 
Contracts Manager 
Contract Specialist 
Buyer 

Procurement Analyst 
Contract Negotiator 
Industrial Specialist 
Production Supervisor 


—k 
mA SB RP aN A A 


Total Number of Interviews 52 


[Source: Procurement Activity Interviews] 


TABLE 33 


PROCUREMENT ACTIVITY PERSONNEL 
EXPERIENCE LEVEL 


YEARS EXPERIENCE NUMBER 


Less Than 2 Years 
2 to 5 Years 

5 to 10 Years 
Over 10 Years 4 


- © WN 


Total Number of interviews 52 


[Source: Procurement Activity Interviews] 
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TABLE 34 


USEFULNESS OF PREAWARD SURVEYS 


DURING SOURCE SELECTION 

RANK 

0 No Use 

1 

2 

S 

4 

5 Moderately Useful 

6 

7 

8 

9 


10 Very Useful 
Don't Know or Not Applicable 


Weighted Average 


[Source: Procurement Activity Interviews] 
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Preaward Survey information was generally found to be useful during the source selection process. Those 
that gave Preaward Surveys high marks, stated that Preaward Survey information was useful when making 
a responsibility determination on Small and Disadvantage Small Businesses. Although the data suggest 
that Surveys are useful, several respondents gave Preaward Survey information a relatively low mark. In 
several of these cases, respondents stated that they rarely request and use Preaward Surveys because the 
reports take too long to get. One other Government official who gave this area a low mark, stated that 
Preaward Surveys are effective for contractor responsibility determination, but not always useful for “Best 
Value” determinations. 

Procuring Activity personnel were then asked to list the frequency with which they award a contract in 
the face of Negative Preaward Survey. The responses to this question are depicted in Table 35. The 
weighted average of all responses 1s 15.5 percent with most responses grouped between zero and 20 
percent. However, one PCO stated they override the Negative Preaward Survey recommendation 80 
percent of the time when poor delivery performance is the only negative finding in the Survey and the 
requirement is not urgent [Ref. 71]. 

The next question asked Procuring Activity personnel to provide the frequency with which they notify 
CAOs of contracts awarded in the face of a Negative Preaward Survey. The responses to this question are 
contained in Table 36. These data reflect that Procuring Activities provide notification to CAOs 
approximately 62 percent of the time. However, these data reflect a wide range of responses. 
Approximately 42 percent of respondents stated that they provide notice to CAOs 100 percent of the time, 
while 18 percent of respondents stated that they never provide notice of overridden Negative Preaward 
Surveys. 

The next question asked Procuring Activity personnel to provide the percentage of time they felt 
contracts awarded in the face of a Negative Preaward Survey were properly justified. Their responses are 
summarized in Table 37. These data indicate that Procuring Activity personnel feel that an overwhelming 
95.5 percent of contracts awarded under those circumstances are fully justified. Several respondents stated 


that the Federal Acquisition Regulation (FAR) requires the contracting officer to award contracts only to 
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TABLE 35 


FREQUENCY PROCURING CONTRACTING OFFICERS 
OVERRIDE NEGATIVE PREAWARD SURVEYS 


FREQUENCY NUMBER 


Co 


0 Percent 
10 Percent 
20 Percent 
30 Percent 
40 Percent 
50 Percent 
60 Percent 
70 Percent 
80 Percent 
90 Percent 
100 Percent 
Don't Know or Not Applicable 


NO 
NO 


mood ODO OW OO W WN 


Weighted Average 15.5% 


(Source: Procurement Activity Interviews] 
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TABLE 36 


PROCUREMENT ACTIVITY NOTIFICATION OF 
OVERRIDDEN NEGATIVE PREAWARD SURVEYS 


FREQUENCY NUMBER 


0 Percent 
10 Percent 
20 Percent 
30 Percent 
40 Percent 
50 Percent 
60 Percent 
70 Percent 
80 Percent 
90 Percent 
100 Percent 
Don't Know or Not Applicable 


OAN WOON ONNN A 


— —. 


Weighted Average 61.8% 


[Source: Procurement Activitiy Interviews] 


TABLE 37 


PERCENT OF PROCURING CONTRACTING OFFICER 
OVERRIDES JUSTIFIED 


FREQUENCY NUMBER 


0 Percent 
10 Percent 
20 Percent 
30 Percent 
40 Percent 
50 Percent 
60 Percent 
70 Percent 
80 Percent 
90 Percent 
100 Percent 
Don't Know or Not Applicable 


ON" OO CO 00 0 0 


— NO 
co NO 


Weighted Average 95.5% 


[Source: Procurement Activity Interviews] 
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responsible contractors. These respondents stated that they would not award to a contractor they found 
non-responsible, unless the Small Business Administration issued a Certificate of Competency. 

The next three questions ask Procuring Activity personnel to provide the top three reasons they give 
for overriding Negative Preaward Survey recommendations and the percent of contracts completed 
satisfactory under those circumstances. For the purpose of these questions, satisfactory contract 
performance was defined as no or minimal problems noted during contract performance. The responses to 
these questions are contained in Tables 38, 39, and 40. As indicated in Table 38, Procuring Activity 
personnel believe that the issuance of a SBA Certificate of Competency was the most frequent reason for 
overriding a “No Award” recommendation and that 75 percent of those contracts are completed 
satisfactorily. Responses to all three questions were taken into account to determine the second and third 
most frequent reasons for overriding a Negative recommendation. The data in Tables 38, 39, and 40 
indicate that “No Other Source” and the “Contractor performed satisfactorily on recent contracts” were the 
second and third most frequent reasons cited for overriding a Negative Preaward Survey. When viewed in 
the aggregate, these data indicate that Procuring Activity personnel believe that approximately 79 percent 
of contracts awarded in the face of a Negative Preaward Survey are completed successfully. 

To gain insight to the causes of Negative Preaward Surveys, Procuring Activity personnel were asked 
to rank the 5 most common factors causing a Negative recommendation. The responses to this question are 
summarized in Table 41. A rank of 1 indicates that the respondent felt that this factor was the most 
common reason for a Negative recommendation, while a rank of 5 indicated that the respondent felt this 
was the fifth most common reason. Values were then assigned to each ranking to determine a weighted 
score and overall rank for each factor. Number | Rankings received the highest value of 5, while Number 
5 Rankings received the lowest value of 1 (X,=5, X,=4, X,;=3, X,=2 and X,=1). As depicted in this Table, 
production was cited as the most common reason contractors receive a Negative Preaward Survey. Of 
note, these data reflect that production, financial, quality assurance, and technical account for over 90 


percent of all Negative Preaward Surveys . 
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TABLE 38 


MOST FREQUENT REASON PROCURING CONTRACTING OFFICER 
OVERRIDES NEGATIVE PREAWARD SURVEYS & PERCENT OF 
CONTRACTS COMPLETED SATISFACTORILY UNDER THOSE CIRCUMSTANCES 


PCT COMPLETED 
REASON NUMBER SATISFACTORILY 
No other source/sole source contractor 3 81.1% 
Contractor performed satisfactorily on recent contracts 3 75.0% 
Financial condition mitigated with Progress Payments 0 N/A 
Meeting delivery schedule deemed not critical 0 N/A 
PCO willing to accept reduced quality due to urgent need 0 N/A 
SBA issued Certificate of Competency 15 72.1% 
PCO received updated information since PAS issued 3 83.3% 
Contractor successfully rebutted NEG PAS findings 2 80.0% 
Reported deficiencies won't impact proposed contract 4 82.5% 
Maintain industrial base 0 N/A 
Best source of available contractor pool 0 N/A 
Defects/Deficiencies are deemed minor 0 N/A 
Weighted Average All Reasons 77.7% 


[Source: Procurement Activity Interviews] 


TABLE 39 


SECOND MOST FREQUENT REASON PROCURING CONTRACTING OFFICER 
OVERRIDES NEGATIVE PREAWARD SURVEYS & PERCENT OF 
CONTRACTS COMPLETED SATISFACTORILY UNDER THOSE CIRCUMSTANCES 


PCT COMPLETED 
REASON NUMBER SATISFACTORILY 
No other source/sole source contractor 3 86.7% 
Contractor performed satisfactorily on recent contracts 5 76.0% 
Financial condition mitigated with Progress Payments 0 N/A 
Meeting delivery schedule deemed not critical 8) N/A 
PCO willing to accept reduced quality due to urgent need 0 N/A 
SBA issued Certificate of Competency 14 66.0% 
PCO received updated information since PAS issued 9 85.7% 
Contractor successfully rebutted NEG PAS findings 3 65.0% 
Reported deficiencies won't impact proposed contract 5 88.0% 
Maintain industrial base 0 N/A 
Best source of available contractor pool 0 N/A 
Defects/Deficiencies are deemed minor 0 N/A 
Weighted Average All Reasons 76.2% 


[Source: Procurement Activity Interviews] 


TABLE 40 


THIRD MOST FREQUENT REASON PROCURING CONTRACTING OFFICER 
OVERRIDES NEGATIVE PREAWARD SURVEYS & PERCENT OF 
CONTRACTS COMPLETED SATISFACTORILY UNDER THOSE CIRCUMSTANCES 


PCT COMPLETED 
REASON NUMBER - SATISFACTORILY 
No other source/sole source contractor 4 86.7% 
Contractor performed satisfactorily on recent contracts 2 81.4% 
Financial condition mitigated with Progress Payments ) 80.0% 
Meeting delivery schedule deemed not critical 1 UNKNOWN 
PCO willing to accept reduced quality due to urgent need 0 N/A 
SBA issued Certificate of Competency 2 UNKNOWN 
PCO received updated information since PAS issued 4 80.0% 
Contractor successfully rebutted NEG PAS findings 6 85.0% 
Reported deficiencies won't impact proposed contract ) 88.0% 
Maintain industrial base 0 N/A 
Best source of available contractor pool 0 N/A 
Defects/Deficiencies are deemed minor 0 N/A 
Weighted Average All Reasons 83.5% 


[Source: Procurement Activity Interviews] 
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TABLE 41 


MOST COMMON REASONS CONTRACTORS 
RECEIVE NEGATIVE PREAWARD SURVEY 
(Scale: 1=Most Common, 5=Fifth Most Common Reason) 


RAW SCORING WEIGHTED OVERALL 
REASON #1 #2 #3 $4 #5 SCORE RANK 
Production 21 7 9 2 4) 169 1 
Financial 12 10 = ~) 2 147 2 
Quality Assurance 4 12 10 3 3 127 ©) 
Technical 4 13 11 9 =) 126 4 
Accounting System 1 1 2 2 9 28 5 
Non-Responsive 1 1 1 1 10 24 6 
Govt Property Control 0 0 0 0 5 5 7 
Packaging 0 0 0 0 1 1 8 


[Source: Procurement Activity Interviews] 
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The next interview question attempts to determine the correlation of Preaward Survey findings with a 
contractor’s performance under a contract awarded in the face of a Negative Preaward Survey. The 
responses to this question are contained in Table 42. These data indicate that Procuring Activity personnel 
feel that the contractor’s failures or deficiencies line up with associated Negative Preaward Survey findings 
71 percent of the time. 

Procuring Activity personnel usually do not spend much of their time on post-award decisions and 
management. Due to workload constraints, they normally delegate contract administration functions to 
cognizant CAOs. The next two questions attempt to determine if Procuring Activities use Preaward Survey 
information during post-contract administration. Procuring Activity personnel were asked to rank on a 
scale of zero to 10 the usefulness of Negative Preaward Surveys during post-contract administration. A 
rank of zero indicates that the survey was of no use, while a rank of 10 indicates the survey was very 
useful. A rank of 5 indicates the survey was of moderate use. The responses to this question are contained 
in Table 43. As indicated in this Table, the weighted average rank for this question was 6.05. This 
indicates that Negative Preaward Survey information is moderately useful to those Procuring Activity 
personnel who responded to this question.. When asked how they used this information during post- 
contract administration, Procuring Activity personnel provided the following comments and examples: 


1. Negative findings may cause the PCO to request special on-site Government surveillance or 
increased inspection of deficient areas. 


2. Signifies that a Post-Award Orientation Conference may be warranted for this particular contract. 


U2 


Provides specific points of discussion for Post-Award Orientation Conferences to ensure the 
contractor fully understands contract requirements and can meet the delivery schedule. 


4. Depending upon the circumstances, this information may cause the PCO to insert Progress 
Payment provisions in the contract or deny the use of Progress Payments. Negative production, 
past performance and/or financial information may trigger PCO requests for pre-payment audits 
of Progress Payment requests. 

5. Helps PCOs determine the level of post-award surveillance they should request from CAOs. 

6. PCO may request more frequent performance and progress reports from DCMC. 

7.  fthe contractor has quality assurance deficiencies, the PCO may insert a requirement for First 


Article Testing (FAT) into the contract. 
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TABLE 42 


CORRELATION OF CONTRACTOR PERFORMANCE 
WITH NEGATIVE PREAWARD SURVEY FINDINGS 


FREQUENCY NUMBER 


0 Percent 
10 Percent 
20 Percent 
30 Percent 
40 Percent 
50 Percent 
60 Percent 
70 Percent 
80 Percent 
90 Percent 
100 Percent 


pa 
Wh |-NO MD | | O - Oo 


Don't Know or Not Applicable 23 
Weighted Average 71.0% 
[Source: Procurement Activity Interviews] 
TABLE 43 
USEFULNESS OF NEGATIVE PREAWARD SURVEYS 
DURING POST-CONTRACT AWARD ADMINISTRATION 
RANK NUMBER 
0 No Use 6) 
1 1 
Z 3 
3 Z 
4 2 
5 Moderately Useful 3 
6 4 
T Zs 
8 8 
g 4 
10 Very Useful 5 
Don't Know or Not Applicable 15 
Weighted Average 6.05 


[Source: Procurement Activity Interviews] 
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8. 


D: 


Negative Preawards have been used in the past to help document unsatisfactory contractor 
performance during Termination for Default proceedings. They have also proved helpful to 
document anticipatory contract breaches. 


Negative information may signify a need to breakout parts for competitive procurement or to 
qualify additional vendors. 


10. Negative Preaward Survey infomation may lead to a change in acquisiton strategies for follow-on 


procurements. For instance, a “Best Value” approach may be warranted when the lowest bidder 
(or group of lowest bidders) has a demonstrated record of poor performance. 


The next two questions concem the use of positive Preaward Survey information in post-contract 
award planning and decisions. As depicted in Table 44, Procuring Activity personnel use positive 
Preaward Survey information less than half the time for post-contact award decisions. When asked how 
this information is used for post-contract award planning and decisions, Procuring Activity personnel gave 
the following comments and examples: 


> 


Positive information is useful for future contract award decisions, provided the information is not 
too old. 


Positive information about a contractor’s capabilities may allow the PCO to waive First Article 
Testing (FAT) requirements. 


Positive information about a contractor’s accounting system may allow the PCO to insert Progress 
Payment provisions into the contract. 


PCOs may request accelerated delivery after learning about a contractor’s positive production 
capabilities and/or excess production capacity. 


Information is useful in reviewing future delivery extension requests. 


Positive Preaward Survey findings are cited in Post-Negotiation Memorandums to document the 
contractor’s responsibility. 


As with Negative Preaward Survey information, positive information is used as points of 
discussion during Post-Award Orientation Conferences. 


As warranted, positive information is used to reduce or eliminate on-site Government 
surveillance. 


The last group of questions asked Procuring Activity personnel to provide recommended changes to 
the Preaward Survey process to enhance its use during source selection and post-award contract 
administration. The responses to these questions follow: 


eS) 


Need to improve the turnaround time on Preaward Surveys. PCOs and Buyers are reluctant to 
request a Preaward Survey due to the 30 day turnaround. 


With respect to SBA COCs, final responsibility determination should reside at the Procuring 
Activity vice at SBA. PCOs should have the authority to override a COC. 


Disestablish the SBA’s COC Program in its entirety. 
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TABLE 44 


FREQUENCY PROCURING CONTRACTING OFFICERS 
USE POSITVE PREAWARD SURVEYS IN 
POST-AWARD PLANNING DECISIONS 


FREQUENCY NUMBER 


O Percent 
10 Percent 
20 Percent 
30 Percent 
40 Percent 
50 Percent 
60 Percent 
70 Percent 
80 Percent 
90 Percent 
100 Percent 
Don't Know or Not Applicable 


aAwnoowaon - fA Oo 


NO 
oO 


Weighted Average 45.0% 


[Source: Procurement Activity Interviews] 
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10. 


Lae 


12. 


14. 


lS: 


Preaward Surveys should include more analysis vice just a presentation of facts and data. 


In addition to providing an “Award” or “No Award” recommendation, Preaward Surveys should 
include a risk rating on all evaluated factors. A risk rating of high, medium or low would be 
helpful during source selection and “Best Value” decisions. 


Preaward Surveys should contain more input from Quality Assurance Representatives. 


PCOs and Buyers should tailor their Preaward Survey requests more than they do now. They 
should only request DCMC to evaluate factors that they need. 


Preaward Surveys should include more past performance history to facilitate “Best Value” 
decisions. 


In lieu of performing multiple Preaward Surveys on the same contractor, recommend DCMC 
perform one Preaward Survey on that contractor each year and provide updates on an as needed 
basis. 


The SBA COC process is entirely too lengthy. To shorten the tumaround time, an SBA 
representative should be represented on the DCMC Preaward Survey team. If this is not feasible, 
SBA should be required to conduct a concurrent review with DCMC’s Preaward Survey team. 


Negative Preaward Survey findings should be red flags for guiding contract administration efforts. 
The ACO needs a copy of the Preaward Survey and should work with the PCO and other 
functional specialists to ensure the negative findings are corrected by the contractor. 


The SBA phone system needs updating. It takes entirely too long to get in contact with personnel 
working on COCs. 


. More emphasis should be placed on citing the reasons for delinquencies vice just reporting 


statistics. 


The Defense Contract Audit Agency (DCAA) should improve their turnaround time on 
accounting system evaluations. 


DOD should establish a centralized data base of Preaward Survey reports that can be accessed via 
the Internet. PCOs could then access this information to see if a current Survey is on file. Ifa 
current Survey is on file, the PCO could use that information to assist with a responsibility 
determination. If a current Survey is not on file, the PCO could request an update to an existing 
Survey or request a new Survey depending upon their needs. 


. When procuring highly technical or complex items, CAOs should ensure that cognizant Technical 


Activity personnel are invited to participate in the Preaward Survey. 


Ue 


IV. DATA ANALYSIS AND INTERPRETATION 


A. GENERAL 

This chapter provides an analysis and interpretation of data contained in Chapter III. The purpose of 
this analysis is to determine the Department of Defense’s (DOD’s) experience with contracts awarded in 
the face of a Negative Preaward Survey and assess the effectiveness of the Preaward Survey process as a 
source selection and risk management tool. This analysis will be accomplished by grouping data collected 
from Defense Contract Management Command (DCMC) Preaward Survey files, Contract Administration 
Office (CAO) interviews and Procuring Activity interviews around the individual research questions. 

This Chapter will reference Tables contained in Chapter III to facilitate analysis and interpretation of 
the data. Ifreferenced herein, Tables 1 through 44 can be found in Chapter III. Tables 1 through 44 
provide data collected from individual sources. Tables 45 and above are presented in this Chapter. In 
general, the Tables presented in this Chapter group data collected from various sources around individual 
research questions. 

When available, this study used data collected from Preaward Survey files, CAO interviews and 
Procuring Activity interviews to answer the research questions. Data from all three sources were used to 
conduct an analysis of the primary causes of Negative Preaward Surveys. Data from CAO and Procuring 
Activity interviews were used to analyze the reasons PCOs give for overriding a Negative Preaward Survey 
and DOD’s experience with contracts awarded under those circumstances. The determination of the 
relative percentages of Negative Preaward Surveys was based solely on DCMC records and Preaward 
Survey files. 

As discussed in Chapter III, the same or similar interview questions were given to CAO and Procuring 
Activity personnel to gain their perspective on Preaward Survey processes. These questions were designed 
to gather data based upon the respondent’s beliefs, personal experience, first hand knowledge and/or 


second hand knowledge. 


Although CAO and Procuring Activity personnel work within the same DOD Acquisition System, their 
roles and responsibilities are different. Their responses may be based upon a narrow scope, limited 
experience and/or personal job bias. Responses under these circumstances usually provide a wider range 
of data than data obtained from records and files. Given the above, DCMC records are presumed to be 
correct if there is a wide disparity between data obtained from DCMC files, CAO interviews and Procuring 
Activity interviews. If DCMC files were not available, further analysis was conducted to determine 
possible reasons for any disparity in source data. 


B. ANALYSIS OF PRIMARY CAUSES OF NEGATIVE PREAWARD SURVEY 
RECOMMENDATIONS 


The purpose of this section is to determine the primary reasons contractors receive Negative Preaward 
Survey recommendations. Data collected from CAO interviews, Procuring Activity interviews and DCMC 
Preaward files were used to conduct this analysis. 

Prior to presenting this analysis, it is important to understand the relative magnitude of this issue. As 
reflected in Tables 1 and 2, the Defense Contract Management Command (DCMC) performed 7,124 
Preaward Surveys between March 1995 and December 1996. Over this 22 month period, 1,640 contractors 
received a Negative Preaward Survey recommendation from DCMC. These data reflect that approximately 
23 percent of all Preaward Surveys performed resulted in a “No Award” recommendation. However, 
further review of the data presented in Tables 4 through 8 indicate that the percentage of Negative Surveys 
is much higher at some DCMCs. For instance, DCMC St. Louis reported a 42 percent Negative Survey 
rate during a ten month period in 1995 and DCMC Birmingham reported a similar rate for calendar year 
1996. Additionally, a high rate of Negative Survey recommendations was noted during areview of 
Preaward Survey files at DCMC Philadelphia. As reflected in Table 16, approximately 45 percent of the 
152 Preaward Surveys performed by DCMC Philadelphia during calendar year 1995 resulted in a “No 
Award” recommendation. 

The relatively high percentage of “No Award” recommendations issued by DCMC could have 


positive implications for the DOD Acquisition Process. When conducted properly, the Preaward Survey 


74 


process helps ensure that only responsible and capable contractors receive Government contracts. 
Awarding a contract to a contractor who is not deemed responsible could affect mission readiness if the 
contractor subsequently experiences performance problems. In these cases, the issuance of a Negative 
Preaward Survey supports DOD’s mission and, ultimately, reduces total acquisition and life cycle costs. 

When a PCO does not have sufficient knowledge of a potential contractor’s capabilities, he or she can 
request DCMC perform a Preaward Survey to assess whether the prospective contractor has the requisite 
capability to perform. Depending upon the information needed, PCOs can request that DCMC evaluate the 
contractor’s capabilities in the following areas: technical, production, quality assurance, accounting 
system, Government Property control, transportation, packaging, security, safety, environmental 
considerations and any other factor deemed crucial to the acquisition. 

In order to determine the most frequently requested evaluation factors, data were collected from 
DCMC Philadelphia’s 1995 Preaward Survey files. As reflected in Table 15, these data indicate that the 
most frequently requested evaluation factor is production capability, followed by technical capability, 
quality assurance capability and financial capability. Additionally, it should be noted that the above factors 
account for over 78 percent of all factors evaluated. Given the above, these data suggest that PCOs 
consider these four capabilities to be the most crucial for successful contract performance. It should be 
noted that DCMC files provided the only source of data for this portion of the analysis. This information 
was not requested during CAO and Procuring Activity personnel interviews because it was felt that a 
record review of actual Preaward Survey requests would provide the most valid data. 

Data concerning the primary causes of Negative Preaward Surveys were collected from DCMC 
Philadelphia Preaward Survey files, CAO personnel interviews and Procuring Activity personnel 
interviews. The results of this research are depicted in Tables 15, 29 and 41, respectively. To facilitate 
comparison and analysis, the data from these three sources are summarized in Table 45. As depicted in this 
Table, all three sources agree that the top three factors causing Negative Preaward Surveys were 
production, financial and quality assurance. DCMC files indicate that Walsh-Healey determinations and 


technical were the fourth and fifth factors, respectively, causing Negative Preaward Surveys. Whereas, 
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TABLE 45 


PRIMARY CAUSES OF NEGATIVE PREAWARD SURVEYS 


DCMC CAO PROCURING 
PAS FILE INTERVIEW ACTIVITY 

REASON RANK RANK RANK 
Production 1 Tied 1 1 
Financial 1 Tied 2 Z 
Quality Assurance 3 3 3 
Technical 5 4 4 
Other: Walsh-Healey 4 N/A N/A 
Non-Responsive N/A 5 6 
Accounting System N/A 6 5 
Packaging N/A vf 8 
Govt Property Control N/A 8 fi 


[Source: Tables 15, 29 & 41] 
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both CAO and Procuring Activity personnel ranked technical as the fourth factor. It should be noted that 
as of 01 October 1995, the Federal Acquisition Streamlining Act (FASA) eliminated the Walsh-Healey Act 
requirement that a contractor be either a “regular dealer” or a “manufacturer” to be eligible for a 
Government supply contract in excess of $10,000. [Ref. 115:p.22.604] Discounting the Walsh-Healey 
ranking in the DCMC files, these data indicate that the top four reasons for Negative Preaward Surveys are 
production, financial, quality assurance and technical. As indicated in Table 15, these four factors account 
for over 90 percent of all Negative Preaward Survey findings. The fact that the top four factors requested 
for evaluation are also the top four factors causing Negative Preaward Surveys indicate that Preaward 
Survey resources are potentially being effectively utilized. 

In addition to determining the major causes of Negative Preaward Surveys, this study collected data on 
various factors listed in DCMC Preaward Survey files to ascertain if there were any trends associated with 
Negative Preaward Surveys. During this review, data were collected on the dollar value of proposed 
contracts, low dollar value Preaward Survey requests, business size, proposed contract type and the 
percentage of proposed contracts with financing arrangements. The data collected during this record 
review are presented in Tables 10 through 14. 

With respect to the dollar value of proposed contracts, the data in Table 10 indicate that both the 
average and median dollar value of proposed contracts listed in Negative Preaward Surveys were 
significantly lower than the average and median dollar values listed in Positive Preaward Surveys. Of note, 
the median dollar value listed in Negative Preaward Surveys was just one third of the value listed in 
Positive Preaward Surveys. Although there is no supporting data, the researcher postulates that the reason 
for this disparity may be due to that fact that less established small businesses tend to bid on the lower 
value contracts. These type businesses typically have less capabilities, less experience and limited financial 
resources. As such, these type businesses tend to experience more difficulties during contract performance. 

As depicted in Table 10, over 22 percent of Negative Preaward Surveys were caused by prospective 
contractors declining to participate in the Preaward Survey. A prospective contractor is deemed non- 


responsive when they decline to participate in the Survey, withdraw their bid or proposal, fail to extend 
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their bid or fail to furnish requested documentation. When this occurs, the prospective contractor fails to 
affirmatively demonstrate the general standards of responsibility cited in FAR 9.104-1 [Ref. 115:p. 9-1]. 
The data contained in Table 10 reflect that the average and median dollar value of proposed contracts for 
these type Negative Surveys is noticeably lower than the overall median dollar value for all Negative 
Preaward Surveys. Review of Preaward files revealed several cases of prospective contractors 
withdrawing their bid on these low dollar value contracts after learning that the PCO had requested a 
Preaward Survey. These contractors stated that the low dollar value of the proposed contract did not 
warrant the time, effort and resources required to participate in the Preaward Survey. Accordingly, those 
contractors chose to either formally withdraw their bid or decline to participate. In another instance, a 
prospective contractor admitted, after learning about a pending Preaward Survey, that they had not yet 
remedied all their delivery delinquencies. Knowing that their past delivery performance did not meet 
Government requirements, the prospective contractor withdrew their bid to avoid the time and expense of 
going through a Preaward Survey that would find them non-responsible. Although, not substantiated from 
documented files, several DCMC personnel stated that the high rate of withdrawn bids is due to 
indiscriminate bidding by inexperienced small businesses. These contractors perform a cursory review of 
proposed contract requirements and submit a bid without performing a thorough cost analysis or proposal 
review. After they submit their bid they start to learn what efforts are required to perform. At that point, 
the contractor realizes that they do not have the requisite capabilities and/or that they substantially 
underestimated the cost to perform. 

One additional trend was noted with respect to low dollar value contracts listed in Negative Preaward 
Surveys. Low dollar contracts are defined as contracts having a proposed value of less than $25 thousand. 
Per the Federal Acquisition Regulation, PCOs should not request a low dollar value Preaward Survey 
unless circumstances justify its cost [Ref. 115:p. 9-3] As reflected in Table 11, approximately 21 percent 
of all Preaward Surveys conducted at DCMC Philadelphia during 1995 had a proposed dollar value less 
than $25 thousand. Of note, almost twice as many low value Preaward Surveys were Negative than 


Positive. This disparity may be attributed to the contract requirements found in many low value Preaward 
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Survey requests. As indicated in Table 11, over 43 percent of low dollar value Survey requests were for 
parts or material destined for critical end item applications. Typically, the manufacturing requirements for 
these procurements entail complex processes and require sophisticated inspection and quality assurance 
systems. The complex nature of contract requirements may explain the increased number of Negative 
Preaward Survey recommendations in this area. Additionally, if non-qualified small businesses bid on 
these contracts, they may receive a higher percentage of “No Award” recommendations due to their 
limited experience with contracts requiring stringent quality standards and controls. 

During the Preaward Survey file review, data were collected on the size of businesses being surveyed 
and the relative percentage of Negative Preaward Surveys issued to both large and small businesses. 
Contractors report the size of their business as either “Large” or “Small” on all bids and proposals for 
Government contracts. Normally, PCOs do not challenge a contractor’s business size claim unless they 
have specific knowledge of a discrepancy in this area. The size of the prospective contractor’s business is 
then listed on the formal Preaward Survey request. The results of this record review are contained in Table 
12. As reflected in this Table, approximately 75 percent of all Preaward Surveys performed by DCMC 
Philadelphia in 1995 were performed on Smail Businesses. The high percentage of Surveys on small 
businesses is not surprising when viewed in light of why PCOs request Preaward Surveys. PCOs request 
Surveys when they do not have sufficient knowledge of a potential contractor’s capabilities to perform ona 
Government contract. Typically, PCOs have sufficient knowledge at hand to make a responsibility 
determination on larger, more established businesses. Often, these businesses are currently performing on 
Government contracts and/or have a proven performance record on previous Government contracts. Under 
these circumstances, PCOs tend to use first hand knowledge about a contractor’s capabilities in lieu of 
obtaining this information via a formal Preaward Survey. As noted by several PCOs, this practice reduces 
Procurement Administrative Lead Time (PALT). Additionally, this practice reduces the number of 
Preaward Surveys performed on “no or low risk” contractors. Given the above, these data suggest that 
PCOs are requesting Preaward Surveys only when needed and that DCMC Preaward Survey resources are 


being utilized effectively. 


ie 


In addition to containing information on the number of Preaward Surveys performed on large and 
small businesses, Table 12 contains data on the relative percentage of Negative recommendations issued to 
large and small businesses. These data indicate that small businesses received a disproportionate 
percentage of “No Award” recommendations. Small businesses received Negative Preaward Surveys 
approximately 38 percent of the time, whereas large businesses received Negative Surveys approximately 7 
percent of the time. As discussed previously, this disparity may be attributed in part to contractor 
inexperience, indiscriminate bidding practices, the high number of low dollar contracts with critical end 
item applications, and the high percentage of contractors declining to participate in low dollar value 
Preaward Surveys. 

During the DCMC Preaward Survey file review, data were collected on whether the proposed contract 
was either a Fixed-Price type contract or a Cost-Reimbursement type contract. The purpose of this data 
collection was to determine if the contract type had any bearing on the number of Negative Preaward 
Surveys issued. The results of this research are summarized in Table 13. These data indicate that 
approximately 95 percent of all Preaward Surveys were performed on contractors vying for Fixed-Price 
type contracts and that Preaward Surveys were performed infrequently on contractors vying for Cost- 
Reimbursement type contracts. Additionally, these data indicate that Negative recommendations were 
rarely given on Surveys containing a Cost-Reimbursement type contract, whereas surveys on Fixed-Price 
type contracts mirrored the overall Negative recommendation rate. The reason for the higher rate of 
Positive recommendations on Cost-Reimbursement type contracts is probably attributed to the unique 
nature and requirements associated with these type contracts. Cost-Reimbursement type contracts are 
usually issued for large dollar value, highly complex and/or risky developmental requirements. In addition 
to possessing the requisite technical and management capabilities, prospective contractors must have an 
adequate accounting system to be eligible for award of a Cost-Reimbursement type contract. As such, 
larger sized and more established companies tend to propose on these type efforts. The larger, more 
established companies, by virture of their experience and resources, tend to have better contract 


performance records. 
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Lastly, data were collected on whether the proposed contract contained Progress Payment provisions. 
These data were collected to determine if the inclusion of financing arrangements in the contract had any 
bearing on the Preaward Survey recommendation. These data are summarized in Table 14. As reflected in 
this Table, approximately 15 percent of all Preaward Surveys performed by DCMC Philadelphia in 1995 
contained financing provisions. The relatively low percentage of contracts with Progress Payment 
provisions is not surprising when viewed in conjunction with the fact that the median dollar value of all 
Preaward Surveys was approximately $127 thousand. Typically, PCOs do not include Progress Payment 
provisions in low dollar value contracts or contracts with relatively short periods of performance. As 
reflected in Table 14, the rate of Positive recommendations on Surveys containing financial arrangements 
was 8.6 percent, whereas the rate of Negative recommendations was 6.6 percent of all Preaward Surveys 
conducted. These data indicate that the inclusion of financing arrangements in a proposed contract had no 
or little bearing on the outcome of a Preaward Survey recommendation. 


C. ANALYSIS OF REASONS WHY PROCURING CONTRACTING OFFICERS OVERRIDE 
NEGATIVE PREAWARD SURVEYS 


The purpose of this section is to determine the primary reasons why Procuring Contracting Officers 
(PCOs) award a contract in the face of a Negative Preaward Survey recommendation. Data collected from 
Contract Administration Office (CAO) and Procuring Activity personnel interviews will be used to 
facilitate this analysis. 

Prior to conducting an analysis on the reasons why PCOs override Negative Preaward Surveys, it is 
important to understand the frequency with which this event occurs. To gain insight into the magnitude of 
this issue, CAO and Procuring Activity personnel were interviewed to determine the frequency with which 
PCOs override DCMC’s “No Award” recommendations. CAO and Procuring Activity responses are 
contained in Tables 24 and 35, respectively. Additionally, data were obtained via follow-up interviews with 
Procuring Activity personnel to determine if any of the 68 Negative Preaward Surveys issued by DCMC 
Philadelphia in 1995 were overridden by PCOs. These data are presented in Table 16. To facilitate 


comparison and analysis, the data from these three sources are summarized in Table 46. As reflected in 
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TABLE 46 


FREQUENCY WITH WHICH PROCURING CONTRACTING OFFICERS 
OVERRIDE NEGATIVE PREAWARD SURVEYS 


SOURCE PCT OVERRIDEN 
Actual Overrides of DCMC Philadelphia's 13.2% 

1995 Negative Preaward Surveys 
CAO Personnel Interviews 31.9% 
Procuring Activity Personnel Interviews NSIS 76 
AVERAGE OF 3 SOURCES 20.2% 


[Source: Tables 16, 24 & 35] 
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Table 46, the data obtained from Procuring Activity personnel interviews and actual contract award records 
at Procuring Activities are consistent. These data indicate Procuring Activity personnel believe that PCOs 
override Negative Preaward Surveys approximately 15 to 16 percent of the time, while actual overrides of 
Negative Preaward Surveys conducted in 1995 occurred at a frequency of 13.2 percent. The consistency of 
these data is not surprising since Procuring Activity personnel would have the greatest insight into contract 
awards and the circumstances surrounding those awards. Conversely, CAO personnel have relatively little 
insight into Procuring Activity’s source selection process after issuance of a Preaward Survey. As reflected 
in Table 23, CAO personnel stated that they received notice of overridden contracts less than 7 percent of 
the time. Given the lack of Preaward Survey feedback to CAO personnel, there is a strong possibility that 
CAO responses may have been biased by limited experience, second hand information and/or the general 
lack of statistical data on this subject. Accordingly, the researcher believes that the Procuring Activity 
responses and the actual percentage of overridden Negative Surveys provide more representative data than 
CAO interview responses. Discounting CAO interview responses, these data indicate that PCOs override 
Negative Preaward Surveys approximately 15 percent of the time. 

PCOs have a fiduciary responsibility to award contracts only to responsible contractors and to make 
the best business decision given the circumstances surrounding any given acquisition. The following 
section provides an analysis of why PCOs occasionally override DCMC’s “No Award” recommendations. 
Data collected from CAO personnel interviews, Procuring Activity interviews and Procuring Activity files 
of overridden contracts will be used during the conduct of this analysis. 

During the interview phase of this study, CAO and Procuring Activity personnel were asked to rank 
the three top reasons why PCOs override Negative Preaward Surveys. The responses to these questions are 
contained in Tables 26, 27, 28, 38, 39 and 40. These Tables provide individual responses to the first, 
second and third most frequent reasons for overriding a “No Award” recommendation. To facilitate 
analysis, the three individual responses were grouped together by activity and provided a weighted ranking 
to determine the relative frequency of a given response. The CAO and Procuring Activity response 


summaries are provided in Tables 47 and 48, respectively. 
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In addition to obtaining data via CAO and Procuring Activity interviews, follow-up research was 
conducted on the nine Negative Preaward Surveys that were overridden by PCOs in 1995. The reasons for 
awarding these contracts in the face of a Negative Preaward Survey were obtained from follow-up 
interviews with cognizant Procuring Activity personnel. Their responses were based upon review of 
associated overridden contract award files. The specific reasons PCOs cited for awarding these contacts 
are contained in Table 17. 

A summary of the data collected from the various sources is provided in Table 49. This Table provides 
CAO and Procuring Activity weighted scores and rankings for the various reasons for awarding a contract 
in the face of a Negative Preaward Survey. Additionally, this table lists the specific reasons given by PCOs 
for overriding nine Negative Surveys issued by DCMC Philadelphia in 1995. It should be noted however, 
that there were a total of just 13 reasons cited for these nine overrides. Although, these reasons provide 
insight into the source selection process, it was felt that this sample was too small to establish a meaningful 
ranking for this data source. Therefore, the following analysis will rely more heavily upon the responses 
cited by Procuring Activity and CAO personnel. When there is a discrepancy between these two sources, 
the Procuring Activity data will be presumed to be more representative since personnel assigned to these 
activities have direct involvement with source selection and contract award decisions. 

The data presented in Table 49 clearly indicate that issuance of SBA COCs and sole source awards are 
the number one and two reasons given for overriding Negative Preaward Surveys. Additionally, these data 
suggest that there is a general consensus among CAO and Procuring Activity personnel that “recent 
satisfactory performance” is a frequently cited reason for overriding a Negative Survey. The largest 
disparity in the data concerns the frequency with which PCOs cite “receipt of favorable, updated 
information” as a reason for justifying an award. CAO personnel ranked this reason extremely low, while 
Procuring Activity personnel gave this reason the third highest ranking. In fact, PCOs cited “receipt of 
updated information” as the primary justification for awarding contracts on three of the nine overridden 
contracts identified in this study. The disparity between CAO and Procuring Activity rankings may be 


attributed to the lack of feedback CAO personnel receive on Preaward Surveys and subsequent contract 
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award decisions. Conversely, Procuring Activity personnel have first hand knowledge of contract award 
decisions and the rationale behind those decisions. Discounting the CAO response bias, these data indicate 
that the “Issuance of SBA COCs’, “Sole Source awards” and “Receipt of updated, favorable information “ 
are the three most frequent reasons why PCOs award a contract in the face of a Negative Preaward Survey. 

As indicated above, PCOs are empowered to weigh a multitude of factors and circumstances 
surrounding a particular procurement prior to making a contract award. In order to gain insight into the 
soundness of override decisions, CAO and Procuring Activity personnel were asked to provide the percent 
of overridden contracts that were justified. CAO and Procuring Activity responses are provide in Tables 
25 and 36, respectively. These data indicate that there is wide divergence of opinion between those 
responsible for pre-award contract decisions and those responsible for post-award contract management. 
As indicated in these Tables, CAO personnel feel that 38.5 percent of overrides are properly justified, 
while Procuring Activity personnel feel an overwhelming 95.5 percent of overridden contracts are properly 
justified. The disparity in responses may be due to CAO personnel perceptions regarding the frequency of 
contracts awarded under SBA’s Certificate of Competency (COC) Program and the belief that a high 
percentage of contractors experience performance problems under COC contracts. This assumption is 
supported by data contained in Tables 26, 27 and 28. These data indicate that CAO personnel believe that 
the issuance of SBA COCs is the number one reason for overridden contracts and that the percent of 
contracts completed satisfactorily under those circumstances is approximately 30 percent. The second 
reason for the disparity in data may be attributed to CAO personnel’s lack of insight into the particular 
circumstances surrounding each procurement and the rationale for subsequent contract award. Given the 
potential bias of CAO responses, the researcher believes that Procuring Activity responses summarized in 
Table 37 provide a more representative view of the percentage of overridden contracts that are properly 
justified. 

Procuring Activity personnel are responsible for ensuring contracts are only awarded to capable and 
responsible contractors. As refected in Table 37, Procuring Activity personnel believe that the vast 


majority of contracts awarded in the face of a Negative Preaward Survey are awarded to responsible 
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contractors. On the surface, this high percentage provided by Procuring Activity personnel may be 
considered biased and/or defensive. Some Procuring Activity personnel may have felt that they would be 
putting themselves “on report” if they stated some contract awards were not justified. This bias may have 
entered Procuring Activity responses, however the researcher believes that the vast majority of responses 
were given without bias. This belief is based upon a review of the reasons provided by Procuring Activity 
personnel for overridding Neagative Preaaward Surveys and the soundness behind those decisions. 

As reflected previously in Table 49, the most frequently cited reasons for overriding a Negative 
Preaward Survey are “receipt of a SBA COC”, “Sole Source award’, “receipt of undated, favorable 
information” and “knowledge that the contractor performed satisfactorily on recent, similar contracts”. Per 
the Federal Acquisition Regulation, PCOs are required to award contracts to Small Businesses who receive 
a Certificate of Competency from the Small Business Administration [{Ref. 115:p. 19-33]. Under these 
circumstances, Procuring Activity personnel concurred with DCMC’s “No Award” recommendation but 
were forced to make the award by regulation. Receipt of a COC is then used to “justify” the award. 

During personnel interviews, PCOs voiced a high level of confidence in their contact award decisions, but 
expressed a low level of confidence in SBA COC decisions. This viewpoint is supported by data contained 
in Tables 38, 39 and 40. These data indicate that Procuring Activity personnel believe that only 69 percent 
of all SBA COC issued contracts are completed satisfactorily. Of note, this same viewpoint was discovered 
during the review of the nine overridden contacts issued during 1995. During this review, PCOs stated that 
they would not have awarded contracts to the two contractors who had received SBA COCs. 

The second, third and fourth most frequently cited reasons for overriding a “No Award” 
recommendation are “Sole Source award”, “receipt of undated, favorable information” and “knowledge 
that the contractor performed satisfactorily on recent, similar contracts”. In the case of sole source awards, 
the PCO’s options are extremely limited. Often, the sole source contractor is the only known provider of 
the needed material and services or holds proprietary rights to the product, data or critical manufacturing 
processes. If the Service requirement is urgent, the PCO has no other choice but to award a contract to the 


sole source. As noted in Table 17, one of the nine overridden contracts discovered during this study was 
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awarded to a sole source contractor. In that case, the PCO justified the award by stating it would take too 
long to qualify a second source. Award of a contract under these circumstances is justified by the need to 
satisfy a Service requirement and delivery needs. 

Receipt of updated, favorable information about a contractor’s capabilities is the third most frequently 
cited reason for overriding a Negative Preaward Survey. The DOD business community operates in a fast 
pace, dynamic environment. The circumstances surrounding a particular procurement are not always 
static. Likewise, a prospective contractor’s capabilities may change over time, even in the relatively short 
time between issuance of a Preaward Survey and the subsequent contract award decision. The contractor 
may have received a Negative recommendation based upon its failure or unwillingness to participate in a 
low dollar value Preaward Survey. In these cases, PCOs may obtain information from internal or external 
sources to facilitate a responsibility determination. For instance, commercially available financial reports 
are often used by Procuring Activities to aSsess a prospective contractor’s financial capability. Also, the 
contractor may present updated information to the PCO indicating that a previously noted deficiency had 
been remedied. Under these circumstances, awards are justified based upon the receipt of updated 
information which indicates the contractor is responsible and eligible to receive an award. 

The fourth most frequently cited reason for overriding a “No Award” recommendation is provided by 
PCOs when they possess personal knowledge or information that the contractor performed satisfactorily on 
recent, similar contracts. Under these circumstances the PCO does not necessarily disagree with historical 
past performance information contained in the Negative Preaward Survey. Instead, they provide a greater 
weighting and significance to the fact that the contractor had no problems performing on a recent contract 
requiring the same or similar requirements. Provided there are no significant changes in contract 
requirements and/or contractor capabilities, these overrides appear to be justified due to the low risk of 
contract performance problems and/or default. 

Based upon the above analysis, the researcher concludes that the vast majority of contracts awarded in 
the face of a Negative Preaward Survey are justified either by the particular circumstances surrounding the 


procurement or by SBA COC regulations. Additionally, these data indicate CAO and Procuring Activity 
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personnel have a low level of confidence in SBA’s Certificate of Competency award processes and 
decisions. 


D. ANALYSIS OF DOD’S EXPERIENCE WITH CONTRACTS AWARDED IN THE FACE OF A 
NEGATIVE PREAWARD SURVEY 


This section provides an analysis of DOD’s experience with Contracts awarded in the face of a 
Negative Preaward Survey. Data collected from Contract Administration Office (CAO) and Procuring 
Activity personnel interviews will be used to facilitate this analysis. Additionally, data collected on specific 
contractor performance under overridden contracts will be presented in this section. 

The analysis in the previous section determined the primary reasons why PCOs award a contract in the 
face of a “No Award” recommendation. This analysis concluded that the vast majority of overridden 
contracts appeared to be justified based upon the particular circumstances surrounding the procurement. 
However, the question of whether an award under those circumstances is justified becomes moot when the 
contractor begins performance on the contract. What matters then, is that the contractor performs within 
the terms and conditions of the contract. 

To assess contractor performance under overridden contacts, CAO and Procuring Activity personnel 
were asked to list the percentage of overridden contracts completed satisfactorily under various contract 
award justifications. CAO responses to these questions are contained in Tables 26, 27 and 28, while 
corresponding Procuring Activity responses are presented in Tables 38, 39 and 40. To facilitate analysis, 
these data are summarized in Table 50. As indicated in this Table, there is a large difference of opinion 
between CAO and Procuring Activity responses. CAO personnel believe contracts awarded in the face of a 
Negative Preaward Survey are completed satisfactorily approximately 37 percent of the time, while 
Procuring Activity personnel believe the success rate under these circumstances is approximately 75 
percent. The weighted average of CAO and Procuring Activity responses indicates that approximately 60 
percent of overridden contracts are completed satisfactorily. The researcher believes that the disparity in 
the responses is too great to draw any conclusion with respect to the definitive percentage of overridden 


contracts completed successfully. However, these data do indicate that CAO personnel believe that 
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contracts awarded in the face of a “No Award” decision have greater performance risks. As discussed 
previously, this belief may stem from the limited Preaward Survey feedback CAO personnel receive from 
Procuring Activities and the general lack of insight into contact award decisions after issuance of a 
Preaward Survey. Additionally, the CAO responses may be biased by their belief that just 38.5 percent of 
overridden contracts are justified. Given this negative perception, CAO personnel are more likely to 
believe that overridden contracts are predestined to experience a greater percentage of performance 
problems. Conversely, Procuring Activity perception that approximately 96 percent of all override 
decisions are justified may account for their higher expectations of satisfactory contract performance. 

In addition to the above factors, the different roles and responsibilities of CAO and Procuring Activity 
personnel may have biased their opinion concerning the percent of overridden contracts completed 
satisfactorily. CAO personnel are tasked to be the “eyes and ears” of Procuring Activities. One of their 
primary functions is to monitor contractor performance on a frequent basis. Individuals assigned this 
function become very familiar with contractor capabilities and problems. As such, they are typically 
subjected to all types of performance problems, including minor nuisance problems, on a daily basis. 
Given this knowledge and daily exposure to performance problems, CAO personnel may have 
unknowingly used a more strict grading scale during their responses to this interview question. 
Conversely, Procuring Activity personnel are not typically exposed to the minor or nuisance type 
performance problems experienced by contractors. Procuring Activity personnel are more concerned about 
satisfying a Service requirement for material and/or services and may not care about the contractor’s trials 
and tribulations as long as the required products and/or services are delivered to end users. As such, 
Procuring Activity personnel may have used a more lenient grading scale during their responses. The use 
of strict grading scales by CAO personnel and use of more lenient grading scales by Procuring Activity 
personnel may have contributed to the divergence of data contained in Table 50. 

Although a definitive conclusion cannot be made with respect to the overall percentage of overridden 
contracts completed successfully, the data in Table 50 provide an indication of the relative performance 


risk associated with various override decisions. As reflected in this Table, both CAO and Procuring 


Activity personnel believe that contracts awarded via SBA COCs and sole source justifications experience 
more performance problems and are less likely to be completed satisfactorily. Conversely, Procuring 
Activity and CAO personnel have the greatest confidence in overridden awards when reported Negative 
Preaward deficiencies are believed to have no or little bearing on contract performance. Additionally, 
Procuring Activity responses indicate a high level of confidence when award decisions are based upon the 
receipt of updated favorable information, the contractor’s successful rebuttal of Negative Preaward Survey 
findings and knowledge that the contractor has performed satisfactorily on recent, similar contracts. The 
researcher believes that these data reflect the relative risks associated with various contract award 
decisions and that this information could be used to develop preaward and post-contract award risk 
mitigation strategies. 

In addition to gaining insight on contractor performance from interviews, data were collected on 
specific contractor performance under the nine overridden contracts identified during this study. 
Contractor performance was assessed via follow-up interviews with the DCMC Philadelphia personnel 
responsible for surveillance of those specific contracts. To facilitate analysis, DCMC personnel were asked 
to provide an overall rating of the contractor’s performance and to cite specific performance problems 
noted, if any, under the overridden contract. The results of this study are presented in Tables 18 and 19. As 
reflected in these Tables, 67 percent of the overridden contracts were completed satisfactorily, while just 
23 percent of contracts were rated marginal or unsatisfactory. Specific performance noted under the 3 
contracts rated marginal or unsatisfactory are presented in Table 19. These data indicate that the most 
common factors causing marginal or unsatisfactory performance were late deliveries and quality assurance. 


E. ALTERNATIVES FOR IMPROVING PREAWARD SURVEYS AS A SOURCE SELECTION 
AND RISK MANAGEMENT TOOL 


The purpose of this section is to assess the effectiveness of the Preaward Survey process and to 
provide an analysis of various alternatives for improving Preaward Surveys as a source selection and risk 
management tool. Data collected from CAO and Procuring Activity interviews will be used to facilitate 


this analysis. 
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Prior to addressing specific alternatives for improving the Preaward Survey process, it is useful to 
understand the general impressions of how well the current process is working from those who use it the 
most. When asked this question during interviews, the majority of CAO and Procuring Activity personnel 
felt that the Preaward Survey process was working effectively and that Preaward Surveys were a useful 
tool. 

The relatively small number of major problem areas noted in interview responses tend to support this 
general observation. Although the current process appears to be working well, many CAO and Procuring 
Activity personnel voiced concerns about certain aspects of the Preaward Survey process and offered 
recommended changes. Based upon the number of comments received, it appears that respondents were 
most concerned about the lack of Preaward Survey feedback to CAO personnel, the timeliness of Preaward 
Surveys and the adequacy of SBA Certificate of Competency decisions. To facilitate analysis, CAO and 
Procuring Activity concerns and recommended alternatives for improving the Preaward Survey process are 
grouped into the following categories: 

1. Feedback on Preaward Surveys and Contract Award decisions 

2. Timeliness and Content of Preaward Surveys 


Adequacy of Small Business Administration Certificate of Competency Decisions 
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4. Communication of Preaward Survey Information 

5. Alternative Uses of Preaward Survey Information during Source Selection 

6. Alternative Uses of Preaward Survey Information during Post-Award Administration 
The following sections present an analysis of the six areas outlined above. 

1. Feedback on Preaward Surveys and Contract Award decisions 

Many CAO personnel voiced concern about the lack of feedback they get from PCOs concerning 

the usefulness of Preaward Surveys and the specific award decisions made after issuance of a Preaward 
Survey. As reflected in Table 23, CAO personnel reported that they are notified of overridden Negative 
Preaward Surveys approximately seven percent of the time. It should be noted that this particular issue is 


not new to DOD. Peter M. Evans noted a similar lack of Preaward Survey feedback during a study 
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conducted in 1988. [Ref. P. 105] During the interview phase of this study, one Preaward Survey 
Manager (PASM) stated that DCMC sends a Customer Feedback Card with each Preaward Survey it 
issues. The Feedback Card solicits specific feedback on Preaward Surveys, as well as other DCMC 
products. Although this seems like a good method, the PASM noted that Procuring Activity personnel 
provide feedback on Positive Preaward Surveys less than 20 percent of the time and only rarely do they 
provide feedback on Negative Preaward Surveys. His explanation of why PCOs rarely provide feedback 
on overridden Preaward Surveys is provided below: 
The decision to override a Negative Preaward Survey decision may take significantly longer than normal 
contract awards due to possible involvement with SBA COC procedures, bid protests and/or higher 
management reviews. In these cases, a great deal of the PCO’s time is consumed trying to resolve these 
issues. Given the time that may have lapsed between receipt of the Negative Preaward Survey and final 
contract award, PCOs may have just forgotten about providing feedback to the CAO. Their attention is 
probably focused on making the next of many contract awards. [Ref. 106] 

CAO and Procuring Activity personnel agree that Negative Preaward Surveys contain useful 
information for post-award administration. As noted by one Procurement Analyst, “Negative Preaward 
Survey findings should be red flags for guiding contract administration efforts” [Ref. 20]. The key 
ingredient in this process however, is communication between the Procuring Activity and CAO. CAO 
personnel need to be informed shortly after award of an overridden contract to facilitate development of 
proactive surveillance and risk mitigation plans. Without this notice, CAO personnel may never know that 
a contractor has received an award in the face of a Negative Preaward Survey. As a case in point, CAO 
personnel did not know that contractors received awards on three of the nine overridden contracts 
discovered during this study. Consequently, the CAO took no extraordinary measures on those contracts. 
Performance on two of the three contracts was rated satisfactory by DCMC, however the third contract was 
rated marginal due to delivery problems. 

To improve Preaward Survey feedback, numerous CAO personnel recommended that Procuring 
Activity personnel place the Preaward Survey Serial Number on the face of the contract, regardless of 


whether the Preaward Survey recommendation was positive or negative. This information can then be 


used to determine the required level of surveillance. Contractors who received a “No Award” 
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recommendation might receive increased surveillance in noted weak areas, while contractors who 
demonstrated satisfactory capabilities might receive reduced or no surveillance. Tailoring the surveillance 
level to the perceived requirement helps mitigate performance risk and helps ensure DCMC resources are 
utilized effectively. Several CAO personnel suggested that the Preaward Survey Manager widen the 
internal distribution of Negative Preaward Surveys within the CAO office. Most DCMCs have gone to a 
team based contracts management organization. As a result, this information may be needed by ACOs, 
Contract Administrators, Industrial Specialists, Quality Assurance Specialists, Cost/Price Analysts, 
specialized functional specialists, Team Leaders and DCMC management. 

The specific recommendations discussed above appear to be relatively simple to implement with 
no significant start-up investment other than educating the acquisition work force on the benefits of 
improved Preaward Survey feedback. Additionally, these recommendations do not impose an undue 
administrative burden on either Procuring Activity or CAO personnel. The researcher believes that 
increased levels of Preaward Survey feedback will foster proactive contract administration and enhance the 
effectiveness of post-award contract management. 

An alternate method for obtaining feedback on Preaward Surveys and contract award decisions is 
via the Customer Feedback Card issued with each Preaward Survey. If filled out, this feedback vehicle 
could provide useful information for improving the Preaward Survey process. However, as noted above, 
the response rates on customer service surveys are low. As such, this vehicle would not prove to be a very 
effective or reliable way to communicate information about contract award decisions. Therefore, the 
researcher believes that greater emphasis should be placed on putting Preaward Survey Serial numbers on 
contracts and widening the internal distribution of Negative Preaward Surveys within the CAO. 

2. Timeliness and Content of Preaward Surveys 

With respect to the timeliness of Preaward Surveys, there was a divergence of opinion between 
CAO and Procuring Activity personnel. The Federal Acquisition Regulation states that Procuring Activities 
should allow at least seven working days to perform a Preaward Survey [Ref. 115:p. 9-4 J]. However, 


several CAO personnel stated that seven days was usually not enough time to conduct a Preaward Survey 
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and issue a high quality report to the requesting activity. Additionally, CAO personnel stated that it takes 
approximately 15 days to perform a thorough survey and about 30 days if a secondary survey on a 
subcontractor is required. Accordingly, these same respondents recommended that DCMC should be 
allowed more time to conduct Preaward Surveys. On the other hand, a large number of Procuring Activity 
personnel stated that DCMC needed to improve their turn-around time on Preaward Surveys. Concurrent 
with this recommendation, Procuring Activity personnel stated that the Defense Contract Audit Agency 
(DCAA) needed to improve their turn-around time on Preaward Survey accounting system evaluations. To 
reinforce their comments, several respondents stated that PCOs and Buyers are reluctant to request a 
Preaward Survey due to a lengthy turn-around. 

In addition to pressure from Procuring Activity personnel to improve the timeliness of Preaward 
Surveys, DCMC’s top level report metrics place significant emphasis on Preaward Survey turn-around. 
Several Preaward Survey Managers voiced concern about the emphasis placed on competing a Preaward 
Survey within the requested time frame. These PASMs contend that this metric alone does not address the 
complete product and service they provide to a Procuring Activity. Instead, these PASMs believe that 
equal weighting should be given to the quality of the report. To bolster this position, CAO personnel noted 
that there is little value in a report that meets the report deadline, but fails to provide complete and accurate 
information to facilitate a responsibility determination. Conversely, there can be an argument made by 
Procuring Activity personnel that it doesn’t matter when a report is submitted or whether it is submitted 
within the FAR guidelines of seven days. If the submission doesn’t meet the PCO’s timeline, then you 
have not satisfied your primary customer’s needs. 

Obviously, there are merits to CAO concerns about the time needed to submit a quality product 
and Procuring Activity’s concerns about the timeliness of Preaward Surveys. The researcher believes that 
each request for a Preaward Survey is different, just as each procurement or acquisition is different. Some 
requirements are more complex than others and some will require a longer time to complete. Therefore, 
the application of a standard turn-around time may not take into account the complexity of the required 


effort. Some Surveys can be performed relatively quickly with information available on hand, while others 


98 


may require extensive on-site effort at the contractor’s and subcontractors’ facilities. Given the uniqueness 
of each Preaward Survey and the belief that the customer defines the end product, the researcher elected 
not to collect data on the number of days it took to complete Preaward Surveys. Instead, this section of the 
study will attempt to analyze alternatives which allow DCMC to submit quality Preaward Survey reports 
within the time frame defined and requested by the Procuring Activity. Anything less, would fail to meet 
DCMC customer needs. 

The Procuring Activity’s need for quicker turn-around on Preaward Surveys is probably being 
driven by pressures to reduce Procurement Administrative Lead Time (PALT) and the need to meet a 
Service requirement. On the other hand, the CAO’s need for additional processing time is probably being 
driven by the amount of data that needs to be collected and analyzed, scheduling conflicts with contractors 
and the large coordination required to orchestrate a team wide Survey. Additionally, CAOs and Procuring 
Activities are both constrained by workload demands and limited resources. If a Procuring Activity had 
sufficient money, people and resources, they would be better able to plan their procurements and may be 
able to give DCMC more time to conduct Preaward Surveys. Likewise, if CAOs had more resources, they 
would be able to handle a higher workload and would probably be able to improve the turn-around on 
Preaward Surveys. However, given current fiscal constraints and the growing trend of Government down- 
sizing, it is highly unlikely that DOD will receive additional funding for these purposes. 

In light of finite or dwindling resources at both CAQOs and Procuring Activities, it appears that 
Preaward Survey timeliness is largely a resource allocation issue. Given this premise, the researcher 
believes that possible alternatives for improving the process should look at ways of freeing-up resources at 
both CAOs and Procuring Activities. These resources can then be reallocated to higher priorities or efforts 
which improve the Preaward Survey process. 

One method of freeing-up resources is to improve the efficiency of an existing process. During the 
interview phase of this study, several CAO and Procuring Activity respondents stated that PCOs and 
Buyers should tailor their Preaward Survey requests more than they do now. These statements were based 


upon the belief that Procuring Activity personnel were asking for information that was not germane to the 
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procurement and/or that they did not need every Preaward Survey factor evaluated to facilitate a 
responsibility determination. Formal on-site Preaward Surveys are workload intensive, costly and time 
consuming. When applicable, the tailoring of a Preaward Survey helps DCMC focus on customer needs 
and helps ensure scarce CAO resources are utilized effectively. 

In this same vein, existing Preaward Survey resources could be freed-up by utilizing phone 
Surveys more. DCMC Philadelphia provides an excellent case in point. When current information about a 
contractor's capabilities are already available, the DCMC Philadelphia PASM provides this information via 
a phone call to the PCO or Buyer. This practice satisfies a customer’s need for information in a timely 
manner. Additionally, the use of phone surveys helps avert unnecessary formal on-site surveys and frees- 
up resources to satisfy other customer requirements. Several CAO personnel noted that this practice may 
not be pursued aggressively at other DCMCs, as it reduces reportable workload statistics. Although not 
substantiated, it is possible some DCMCs may elect to conduct formal Surveys almost exclusively to 
increase their workload statistics. 

Within DOD, workload statistics are one of the tools used to determine budget and resource 
allocations. Not counting DCMCs co-located at contractor facilities, Tables 6 and 7 indicate that there 
were eight DCMCs within the continental United States which performed less than 50 Preaward Surveys 
each in 1996. This means that each of these DCMCs must maintain a staff and capabilities to perform 
Preaward Surveys at a rate of less than five per month. Of note, some performed Preaward Surveys at a rate 
of less than two per month. The requirement to conduct these surveys may be valid, however it may not be 
cost effective to maintain associated Preaward Survey overheads at each of these smaller DCMCs. One 
alternative for lowering costs and freeing-up resources is to have the Preaward reporting, management and 
supervision functions for these CAOs moved to a larger DCMC. Only the management and overhead 
responsibility would shift. The functional specialists would still be located at their original DCMC and 
would still service contractors located within their geographic area of responsibility. This alternative 
appears to have merit on the surface, however it is not known whether the personnel performing Preaward 


Survey overhead functions at the smaller DCMCs have other responsibilities which would make them 
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indispensable to that organization. Also, it is not known whether this alternative is cost effective. As this 
alternative has not explored all the possible ramifications associated with this shift of functions, the 
researcher recommends further study in this area. 

In addition to the above DCMC alternatives, there are actions which Procuring Activities may take 
to improve the efficiency of the Preaward Survey process. Per Federal Acquisition Regulation Subpart 
9.106-1, PCOs should only request a Preaward Survey under the following circumstances: 

A Preaward Survey is normally required only when the information on hand or readily available to the 
contracting officer is not sufficient to make a determination regarding responsibility. In addition, if the 
contemplated contract will have a fixed price at or below the simplified acquisition threshold or will 
involve the acquisition of commercial items (see Part 12), the contracting officer should not request a 
preaward survey unless circumstances justify its cost. [Ref. 115:p. 9-3] 
The intent of the above guidance is to promote the effective utilization of available DCMC Preaward 
Survey resources and to reduce the Procurement Administrative Lead Time (PALT) on low dollar 
procurements. However, review of DCMC Preaward Survey files revealed that these guidelines are not 
always followed. As reflected in Table 11, 21 percent of all Preaward Surveys conducted by DCMC 
Philadelphia in 1995 were in support of low dollar value procurements. Out of these requests, PCOs and 
Buyers failed to justify approximately 56 percent of low dollar Survey requests. This study did not attempt 
to determine whether the failure to justify the low dollar value Preaward Survey was merely an oversight 
on the part of the PCO or Buyer. Some of the non-justified Surveys may have been needed to support 
critical end item applications. However, given the relatively high percentage of non-justified Surveys, these 
data indicate that Procuring Activities should screen their low dollar Preaward Survey requests more 
carefully to ensure that the circumstances justify its cost. Better screening of requests will help ensure that 
limited Preaward Survey resources are applied only where needed. Furthermore, the reduction or 
elimination of unjustified Survey requests will free-up resources to improve Preaward Survey turn-around 
and/or satisfy more critical customer needs. 
In addition to addressing low dollar Preaward Survey requests, the FAR guidance cited above 


encourages Procuring Activities to utilize all available resources, including commercially available 


resources, to facilitate a contractor responsibility determination. As circumstances warrant, PCOs and 
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Buyers should aggressively pursue the use of information from other Government activities and 
commercially available resources. The following comments from a DLA Preaward Survey Process Action 
Team (PAT) report issued in 1995 provides an example of the types and breadth of information available 
from commercial resources. 
Commercial evaluations can provide data about the contractor’s facilities and operations, and information 
concerming top management’s business background and the contractor’s corporate relationships. For 
example, Dun and Bradstreet Information Services (D&B) is one of several data suppliers under contract 
with the Government. D&B products include: Supplier Evaluation, Supplier Performance Review, and 
Critical Supplier Analysis (CSA). These reports draw upon information gathered from sources external 
to the company being reviewed, as well as from the company itself. Firms like D&B can provide 
financial statements, risk summaries, quality registration information, and performance measures. 
D&B’s Small Business Sourcing File, for example, provides quick and easy access to information 
concerning more than 250,000 small businesses, including over 39,000 minority and 55,000 women- 
owned businesses. In all, D&B has profiles on over 18.5 million U.S. business locations. [Ref. 111] 

The use of commercially available information, such as financial reporting services, trade publications 
and industry associations, provides numerous benefits to the acquisition process. Procuring Activities may 
receive information from a commercially available source more quickly than they could via a formal, on- 
site Preaward Survey. The DLA Preaward Survey PAT mentioned above, noted that most D&B reports are 
available immediately, while more extensive Critical Suppler Analysis (CSA) reports can be delivered in 
three to five days at a cost of approximately $400 [Ref. 111:p. 6-4]. The resulting reduction in PALT may 
enable the Procuring Activity to satisfy a customer’s needs more quickly. Additionally, the greater use of 
commercially available resources reduces the Preaward Survey workload at DCMC. This reduction in 
workload may lead to improved turn-around on the more complex and critical Preaward Survey requests. 
It should be noted however, that this study did not collect data on how frequently Procuring Activities use 
commercially available information and did not collect data on the adequacy of such information. 
Assuming commercially available information is adequate to facilitate responsibility determinations, the 
researcher concludes that there are tremendous benefits associated with the use of commercially available 
information and that DOD’s use of these resources should be promoted and expanded. 


In addition to using commercially available sources to facilitate responsibility determinations, 


Procuring Activities could reduce their reliance upon DCMC Preaward Surveys by conducting more “Best 
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Value” source selections. During several interviews, Procuring Activity noted that they used to request 
significantly more Preaward Surveys when they were required to award contracts to the lowest qualified 
bidder. Now with emphasis being placed on “Best Value” procurement, these Procuring Activity 
Personnel stated that they no longer have to request a Preaward Survey on the lowest bidder if they 
determine that a higher bidder provides the best value to the Government. This reduction in Preaward 
Surveys was observed during the conduct of this study. As reflected in Tables 1 and 2, DCMC performed 
more Preaward Surveys in a ten month period in 1995 than they did for the entire calendar year of 1996. 
This reduction in the number of Preaward Surveys conducted could be attributed to the general decline in 
Defense budgets and more reliance upon commercially available information sources. Also, the following 
comments from Defense Personnel Support Center’s Chief of Contract Clearance and Oversight attributes 
the decline of Preaward Survey requests at that activity to greater reliance upon alternative information 
sources and “Best Value” procurements. 

Over the last couple of years DPSC has reduced its reliance upon DCMC for Preaward Surveys to the 

point where we rarely request contractor capability information from them. This reduction is attributed 

to greater use of alternative and internal sources of contractor capability information. For instance, our 

PCOs and Buyers now have access to the World Wide Web and frequently obtain financial information 

directly via the Internet from Dun and Bradstreet (D & B). Perhaps the greatest reason for the reduction 

in Preaward Survey requests is due to best value procurement strategies. [Ref. 81] 
As noted during several interviews, best value procurements have reduced Procuring Activity reliance 
upon DCMC for Preaward Survey requests and have reduced the time it takes to make an award. The 
reduction in DCMC’s Preaward Survey workload should be viewed as an opportunity to improve the 
response time on those Survey requests that they do receive. Given the potential reduction to overall 
acquisition costs, the researcher believes that DOD should continue to emphasize best value procurement 
strategies. 

The remainder of this section will present an analysis of alternatives for improving the content of 

Preaward Surveys. During the interview phase of this study, numerous recommendations were received 


concerning ways to improve the content of Preaward Surveys. In some cases, these comments were based 


upon receipt of a Preaward Survey from one source and may not have reflected a need for change 


throughout DCMC. Instead of focusing on isolated cases, the following analysis will focus on comments 
and recommended changes which indicate a process change may be required to enhance the quality and/or 
use of Preaward Surveys. 

One common theme appears to emerge from Procuring Activity comments and recommendations. 
Whether stated explicitly or implied, Procuring Activity personnel wanted additional information added to 
Preaward Surveys to enhance its use during source selection and best value decisions. Along these lines, 
Procuring Activity personnel recommended DCMC provide information on past performance history, trend 
analysis and the reasons behind contractor delinquencies. Several respondents expressed a desire for 
DCMC to include more analysis within the Survey vice just providing a presentation of data. Additionally, 
some Procuring Activity personnel thought that the inclusion of risk ratings on all evaluated factors and the 
addition of a third “Award Not Recommended” recommendation might improve the usefulness of 
Preaward Surveys during source selection and best value decisions. 

Including more past performance history and providing trend analysis on the contractor’s 
performance is well within the existing capabilities of DCMC. This information could be useful to a PCO 
or Buyer during best value decisions, but it is not necessary to have this level of detail in all procurement 
decisions. Instead of requiring DCMC to include this information in all Preaward Surveys, it might be 
more cost effective to provide this information on an as needed basis. As such, it might be beneficial to 
revise the Preaward Survey request form to include information on the proposed basis of award. DCMC 
personnel could then tailor their Survey report based upon how the PCO or Buyer indicated this 
information would be used. For instance, if the PCO indicated this Survey was going to be used in a best 
value source selection, DCMC could include more information on the contractor’s past performance. 
Additionally, under these circumstances DCMC could also provide trend analysis and risk assessments on 
all requested categories. Additionally if the basis for award is best value with particular emphasis on a 
specific capability, DCMC could tailor their Survey to provide increased emphasis on the major factor 
being evaluated during source selection. However, if the Survey is required only for a traditional 


responsibility determination, DCMC could then provide a standard Preaward Survey Report on only those 
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factors requested for evaluation. The key ingredient in this recommended process change is better 
communication of the requirement. DCMC has the requisite capabilities to supply the requested 
information. The PCO or Buyer needs to ensure that their requirements and expectations are clearly 
communicated via the Preaward Survey request. 

Several Contract Administrators recommended DCMC add a third recommendation block titled 
“Award Not Recommended” to the Preaward Survey form. As justification for this recommendation, they 
stated that the performance of some contractors are marginal, but not clear enough to distinguish between 
an “Award” or “No Award” recommendation. This third category could be used to signify that excessive 
Government oversight or assistance beyond what is contractually required may be necessary to ensure 
contract completion. This category would not completely eliminate a contractor from the source selection, 
but would make it easier to make an award based upon “Best Value” to the Government. The researcher 
believes that the information concerning the contractor’s marginal performance is useful information for 
source Selection and should be included in the comments section of the report, regardless of the report’s 
ultimate recommendation. However, there is a danger that adding a third recommendation may add more 
subjectivity to Preaward Surveys. Furthermore, Procuring Activities may become frustrated customers if 
they start to get a lot of noncommittal recommendations. PCOs are looking for a specific recommendation 
from DCMC based upon the experience and judgment of the CAO personnel involved in the Preaward 
Survey process. Submitting an “Award Not Recommended” report may be viewed as pushing work back 
onto the customer. Accordingly, the researcher believes that DCMC should maintain the existing “Award” 
and “No Award” recommendations contained in current Preaward Surveys. These recommendations force 
DCMC to conduct analysis and make a decision, two products PCOs expect out of every Preaward Survey. 

During the interview phase of this study, several CAO personnel expressed concern about the lack 
of specific guidelines for the functional specialists who perform Preaward Surveys. To remedy this 
situation, they recommended that DCMC establish a Preaward Survey Procedures Desk Guide and that 
they provide better training for functional specialists. They stated that the guidelines should make the 


Preaward Survey Process more uniform across the various DCMCs. In turn, the issuance of consistent, 
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high quality reports should give Procuring Activities more confidence in DCMC’s ability to evaluate 
contractors’ capabilities. Apparently this recommendation has been made in the past because DLA is 
currently in the process of publishing a procedures desk guide for functional specialists. As reported by 
DLA’s Deputy Director of Contract Capability and Proposal Analysis, this desk guide is scheduled for 
publication around July 1997 [Ref. 124]. The researcher believes that a desk guide for functional 
specialists will improve the quality of Preaward Surveys as long as DLA and DCMC provide 
commensurate training shortly after the guide is distributed. Another pitfall which should be avoided, is to 
ensure that the desk guide is distributed to all functional specialists, especially those that operate out in the 
field. Based upon the researcher’s personnel experience, those specialists are sometimes the last personnel 
within DCMC to receive news of policy and procedural changes. 

3. Adequacy of Small Business Administration Certificate of Competency Decisions 

This section of the study contains an analysis of the Small Business Administration’s (SBA’s) 
Certificate of Competency (COC) program and the adequacy of its COC award decisions. This analysis 
will be based primarily upon CAO and Procuring Activity personnel interview comments. One limitation 
of this portion of the study is that it presents DOD’s experience only. The study did not attempt to gain 
insights into the COC process from SBA’s or the contractor’s perspective. 

Universally, both CAO and Procuring Activity personnel voiced strong concerns about the SBA’s 
role within the DOD acquisition process and the adequacy of SBA’s COC award decisions. In particular, 
many respondents expressed frustration with SBA’s perceived lenient COC award policy, the lack of 
accountability for SBA’s actions, the apparent duplicative nature of the COC process and the authority 
SBA possesses to override DCMC’s Negative Preaward Survey recommendations and PCO’s non- 
responsibility determinations. Given that sentiment, it came as no surprise that a large number of 
respondents wanted to disestablish the SBA COC Program in its entirety. The following sections will 
analyze the recommendation to disestablish the COC Program as well as those recommendations which 


suggest improvements within the existing frame work of the program. 
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The main argument presented for disestablishing the SBA COC Program is that this Program 
duplicates, in part, the Preaward Survey role performed by DCMC. Additionally, respondents are quick to 
point out that SBA’s track record in assessing prospective contractor capabilities has been historically 
poor. Adding support to this perception is the data contained in Table 50. These data reflect that CAO and 
Procuring Activity personnel believe that just 53 percent of all COC awarded contracts are completed 
satisfactorily. As such, the SBA COC Program costs taxpayers an enormous amount of money to perform 
a mission which often results in additional outlays of time, money and resources. In this era of limited 
resources, the Government can ill afford the cost of duplicative programs. The researcher finds it difficult 
to find any fault with the above argument other then the lack of hard performance data to support 
respondents’ perceptions that almost half of SBA’s COC award decisions result in unsatisfactory contract 
performance. Not withstanding this limitation, the researcher believes that the conduct of a second 
capability survey on a prospective contractor after DCMC has performed a Preaward Survey is duplicative. 
Based upon the results of this study, there is no indication that DCMC’s Preaward Survey process is flawed 
or biased against small businesses. In fact, the data in Table 34 indicate that Procuring Activities find 
DCMC’s Preaward Surveys to be useful during source selection. As such, there is no basis to warrant a 
second survey by SBA. On that argument alone, the researcher recommends disestablishment of the SBA 
COC Program. 

In lieu of performing surveys on small businesses, the researcher believes that SBA should use 
Negative Preaward Survey findings in their smal] business development and training programs. SBA’s 
focus should be to find ways to improve a prospective contractor’s capabilities to the level which makes 
them eligible for future awards. For instance, SBA could provide small businesses training and counsel on 
bid preparation and cost analysis. Prospective contractors could then use this information to determine if 
they possess the requisite capability to perform on a particular contract and whether they have correctly 
estimated the cost of performance. This training could lead to a reduction in the number of indiscriminate 
bids submitted to Procuring Activities. Additionally, SBA’s network of small business specialists could 


work directly with contractors who had received a Negative Preaward Survey. These experienced 
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specialists could provide analysis on the reasons for the “No Award “ recommendation and assist the 
contractor with development of a corrective action plan. 

The remainder of this section will provide an analysis of recommendations which propose 
improvements within the existing framework of the SBA COC Program. Several Procuring Activity 
personnel expressed concern that DOD effectively loses control over contract award decisions after a 
Negative Preaward Survey is forwarded to the SBA. These personnel contend that they are responsible and 
accountable for making good business judgments for the DOD. When the award decision passes to SBA, 
these personnel believe that SBA assumes responsibility but not the corresponding accountability for 
making an award decision. SBA’s involvement after contract award is believed to be minimal at best. 
CAO and Procuring Activity personnel must live with the consequences, both good and bad, of SBA’s 
COC decision. Many respondents stated that this lack of accountability after issuance of a COC may have 
a bearing on the quality of SBA’s COC award decisions and the high percentage of performance problems 
experienced under SBA’s COC Program. In order to tie accountability with responsibility, CAO and 
Procuring Activity personnel recommended that the ultimate award decision reside with the PCO. If this is 
not possible, they recommended that SBA administer all contracts awarded under the SBA COC Program. 
The later recommendation would certainly instill greater accountability within the COC Program, however 
it is doubtful that SBA has the resources or capabilities to administer these contracts. Additionally, such an 
arrangement would establish costly duplicative capabilities, inefficiencies and confusion. Additionally, 
this recommendation would run counter to the “one face” the Government tries to present to industry. 
Therefore, this recommendation is considered not to be a viable alternative by the researcher. However, 
the recommendation to allow DOD veto power on all COC awards does have considerable merit due to the 
arguments presented above. If the SBA COC Program is not disestablished, the researcher believes DOD 
should retain the right to make final award decisions on their requirements. 

Numerous Procuring Activity personnel stated that the SBA COC process is entirely too lengthy. 
To shorten the turn-around time, these respondents recommended that SBA assign a representative to 


DCMC Preaward Survey teams. The SBA representatives could either act as observers or team members 
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to facilitate a concurrent SBA review on prospective small business contractors. This would improve the 
turn-around time on SBA COC decisions and could foster improved communication between SBA COC 
personnel and DCMC. Additionally, the participation in DCMC Preaward Surveys may give SBA greater 
insight into DCMC’s rationale for making “No Award” recommendations. Conversely, SBA’s 
participation may give DCMC personnel insight into SBA’s rationale for making COC award decisions. 
SBA could use this knowledge and experience to improve their small business development and training 
programs. Although this recommendation has considerable merit, SBA may not have the personnel 
resources to assign representatives to each DCMC Preaward Survey team and may not have the resources 
to participate in every Preaward Survey conducted on a small business. Also, SBA may object to this 
arrangement because of fears that their COC personnel may lose their objectivity. Even if SBA does not 
have the resources to populate each DCMC Preaward Survey team, the researcher believes that this 
recommendation has considerable merit. At aminimum, DOD should extend an invitation to SBA to 
participate on selected Preaward Survey teams on a trial or cross training basis. Also, as a standard 
operating procedure, DCMC may want to invite SBA participation in Preaward Surveys conducted on 
known or suspected high risk contractors. 

During the interview phase of this study, CAO and Procuring Activity personnel recommended 
that SBA discuss its rationale with DCMC prior to issuance of aCOC. This gives DCMC an opportunity to 
assess SBA’s rationale for overriding DCMC’s “No Award” recommendation. It also provides DCMC an 
opportunity to provide additional information or clarification of DCMC’s concerns. If SBA does issue a 
COC, several respondents recommended that SBA furnish COC findings and its rationale to the cognizant 
CAO and Procuring Activity. Requiring SBA to forward COC findings and associated justifications could 
instill greater accountability into the COC process. As with Negative Preaward Survey information, COC 
findings could be useful during post-award planning and contract administration. The cost of 
implementation is considered to be low and neither recommendation would impose an undue 
administrative burden on SBA or DOD personnel. Given the above benefits, the researcher recommends 


that DOD pursue implementation of these recommendations. 
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4. Communication of Preaward Survey Information 

This section provides an analysis of recommendations to improve the communication of Preaward 
Survey information. Data collected from CAO and Procuring Activity personnel interviews will be used to 
facilitate this analysis. 

In general, many CAO and Procuring Activity personnel expressed the opinion that the Preaward 
Survey process was working effectively and that Preaward Survey information was useful during source 
selection and post-award administration. However, several respondents noted that improvements could be 
made in how Preaward Survey information is communicated to users. As pointed out by several 
respondents, the value of Preaward Survey reports are limited substantially, if the findings are not 
disseminated to all potential users. 

To increase awareness of Preaward Survey findings, several CAO personnel recommended that 
Preaward Survey Mangers (PASMs) invite all DCMC team members, including ACOs, to participate in 
Preaward Surveys on known or suspected “high risk” contractors. It was felt that DCMC team members 
who did not participate directly in a Preaward Survey were left out of the Preaward Survey process and 
subsequent contract management decisions. These respondents believe that greater participation by ACOs , 
and Quality Assurance Specialists could lead to the development of more effective surveillance and risk 
mitigation plans. Related to the above recommendation, several Procuring Activity personnel 
recommended that CAOs invite cognizant technical activity personnel to participate in Preaward Surveys 
whenever the contract requires manufacture of highly technical or complex items. The knowledge and 
experience of these personnel could help ensure that the prospective contractor fully understands the 
technical aspect of contract requirements and that they have the requisite capabilities to perform the 
proposed contract. In light of the potential benefits noted above and DCMC’s current team approach to 
contract management, the researcher believes that greater team participation in “high risk” Preaward 
Surveys should be encourage throughout DCMC. Additionally, as discussed in a previous section, SBA 


should be encourage to participate as well. 
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Another vehicle which could be used to increase the communication of Preaward Survey findings 
is the Post-Award Orientation Conference. Typically, these conferences are held after issuance of a 
contract to a relatively new defense contractor or when the contract requirements are extremely complex or 
require extensive management and oversight. Besides contractor participation, these conferences may be 
attended by various CAO, Procuring, Technical and Requiring Activity personnel. The purpose of these 
conferences is ensure all parties have a mutual understanding of the contract requirements and the course 
of action to achieve successful contract completion. 

During several interviews, CAO and Procuring Activity personnel noted the benefits of Post- 
Award Orientation Conferences and recommended that DOD increase the use and frequency of Post- 
Award Orientation Conferences whenever a contract is awarded in the face of a Negative Preaward Survey. 
These respondents stated that these conferences should place particular emphasis on ensuring the 
contractor fully understands contract requirements and the requirement to correct deficiencies noted in the 
Negative Preaward Survey. To enhance the effectiveness of this tool, PCOs should ensure appropriate 
CAO team members participate in these conferences. CAO participation is needed to facilitate 
development of appropriate surveillance and risk mitigation plans. If needed, there can be a “Government 
Only” meeting prior to the meeting with the contractor to validate plans, discuss specific areas of concern 
and ensure that the Government presents “one face” to the contractor during the Post-Award Orientation 
Conference. Given the low notification rate of overridden contracts reflected in Table 23, the researcher 
concludes that Post-Award Orientation Conferences are underutilized on contracts awarded in the face of a 
Negative Preaward Survey. Provided the circumstances, potential benefits and risks justify its use, the 
researcher recommends DOD expand the frequency and use of Post-Award Orientation Conferences 
whenever overridden contracts are issued under high risk circumstances. For instance a conference may be 
warranted when a contract is issued under a sole source or SBA COC situation. Whereas, a conference 
may not be warranted when a contractor has demonstrated satisfactory performance on recent, similar 
contracts or when the PCO made the award based upon receipt of updated, favorable information about the 


contractor’s capabilities. 
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One Contract Specialist at a Procuring Activity wanted Preaward Survey information to be more 
accessible to PCOs and Buyers and suggested that DOD establish a centralized database or Government 
Bulletin Board that can be accessed via the Internet. When PCOs and Buyers need information ona 
prospective contractor’s capabilities, they could first search this database to see if a current Preaward 
Survey is on file. If a current Preaward Survey is on file and the planned procurement is similar to the 
efforts evaluated in the Survey, the PCO could use this information to help make a responsibility 
determination. Ifa current Survey is not on file, the PCO could then request an update to an outdated 
Survey or request a new Survey depending upon their needs. 

In addition to providing information on DCMC generated Preaward Surveys, this database could 
be used to capture and disseminate surveys performed by non-DCMC activities. The sharing of survey 
information across agencies, could reduce costly duplication of efforts within the Government and reduce 
the level of Government intrusion placed upon industry. Additionally, use of such a database could lead to 
a reduction in DCMC Preaward Survey workload. DCMC could then take advantage of this reduction in 
workload to improve the quality and timelines of those Survey requests they do receive. 

The cost to implement and maintain a centralized Preaward Survey may be cost prohibitive or the 
perceived need for this tool may be less than some higher priority DOD projects. Also, there may be 
difficulties coordinating efforts across multiple Government Agencies and there may be Preaward Survey 
standardization issues that would have to be resolved prior to implementation. If such a database were 
established, there is a risk that PCOs may make responsibility determinations based upon outdated or non- 
pertinent surveys. As discussed previously, CAO personnel were concemed that PCOs were making 
responsibility determinations based upon General Purpose Surveys vice from contract specific Preaward 
Surveys. To ensure the integrity of the process, PCOs and Buyers would have to carefully assess whether 
current Survey findings accessed via the database could be applied to their particular procurement. If not, 
they would need to request a new Survey from DCMC. Also, there might be concerns about what 
constitutes a “current” survey. The capabilities of a contractor may change over time for better or worse. 


Therefore, to ensure the findings of a particular survey are still applicable, PCOs and Buyers may need to 
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verify this information with the cognizant issuer of the Survey. As with any database, there may be 
concerns about data integrity. If this system is adopted, the system developers should take measures to 
ensure that data entry and maintenance does not create an undue administrative burden on activities which 
are already resource constrained. Not withstanding the above concerns. the researcher believes that the 
potential benefits of such a database are high. Accordingly, the researcher believes that DOD should 
investigate the establishment of a centralized Preaward Survey database to facilitate the communication 
and sharing of Preaward Survey findings among all Government Procurement Activities. 

5. Alternative Uses of Preaward Survey Information During Source Selection 

This section provides an analysis of alternative uses of Preaward Survey information during the 
source Selection stage of the DOD acquisition process. This analysis will be based upon CAO and 
Procuring Activity interview data presented in Chapter III. 

In general, the majority of comments received from Procuring Activity personnel indicate that 
Preaward Surveys are an effective tool for contractor responsibility determinations. As designed, these 
Surveys make a recommendation with respect to a contractor’s capability to perform a given contract. In 
addition to this function, some Preaward Surveys can be used to facilitate a limited source selection 
function. Most Negative Preaward Surveys and some Positive Preaward Surveys are used by PCOs to 
exclude non-responsible and marginal contractors from the pool of contractors being considered for award. 
In this regard, Preaward Surveys told the PCO who not to select. Although useful for these two purposes, 
several PCOs noted that Preaward Surveys could not always be used to help determine which contractor to 
select in a best value procurement. This is especially true with positive Preaward Surveys, where the report 
may not contain any distinguishing comments, either positive or negative, with respect to a contractor’s 
capabilities. 

As discussed previously in this chapter, Procuring activity personnel thought that adding more 
performance history, trend analysis and risk analysis would make Preaward Surveys a better source 
selection tool. Additionally, they recommended DCMC include more analysis of the contractor’s 


strengths and weaknesses vice just presenting data and facts. In addition to the above recommendations 
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from Procuring Activity personnel, several CAO personnel recommended that PCOs assign a higher weight 
to Preaward Survey findings during source selection. In particular, delivery performance and past 
performance history could be used to assist source selection decisions. For good reason, DOD has placed 
tremendous emphasis over the last several years to make contract awards based upon best value to the 
Government. As noted by one Contracts Manager, “award to a marginal contractor based upon lowest 
price does not always yield the best value to the Government, especially if that contractor subsequently 
defaults” [Ref. 81]. Provided the above recommendations are adopted by DCMC, the researcher believes 
that Preaward Surveys could be a viable and useful tool during best value procurement decisions. 
Accordingly, the researcher recommends DCMC implement the above recommendations and look for 
additional ways to enhance the usefulness of Preaward Surveys during source selection. 

6. Alternative Uses of Preaward Survey Information During Post-Award Administration 

This last section provides an analysis of alternative uses of Preaward Survey information during 
post-award administration. This analysis will be based upon CAO and Procuring Activity pesonnel 
interview comments and recommendations. 

As reflected in a number of interview responses, Procuring Activity personnel do not spend much 
time on post-award decisions and management. Several PCOs and Buyers noted the majority of these post- 
award management responsibilities are delegated to DCMC. When PCOs are involved in post-award 
decisions, they tend to use Negative Preaward Survey information more often than positive information. 
Procuring Activity personnel stated they primarily use Negative Preaward Survey information to determine 
the level of post-award surveillance required of DCMC. Relatively few Procuring Activity personnel 
indicated they use this information to mitigate performance risk via changed contract terms and conditions. 
As expected, Procuring Activity use of positive Preaward Survey information after contract award is even 
lower than their use of negative Survey findings. The data in Table 44 reflect that Procuring Activity 
personnel use positive survey information for post-award decisions less than 45 percent of the time. Based 
upon Procuring Activity interview responses, the researcher concludes that Preaward Survey information is 


not being used to its full potential at Procuring Activities. 
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As expected, interview responses from CAO personnel indicated greater use of Preaward Survey 
information than Procuring Activities’ use of this information. When asked to indicate the usefulness of 
Negative Preaward Survey information during post-award administration, the data in Table 31 reflect that 
the majority of CAO personnel found this information useful, while close to half the respondents stated 
this information was very useful. When asked how this information is used, a large percentage of 
respondents stated they use this information to develop post-award surveillance plans. The range of 
responses received from CAO personnel indicate that these individuals understand the potential uses of 
Preaward Survey information. However, this study did not collect data to determine whether this 
knowledge was actually used to develop specific surveillance and risk mitigation plans on overridden 
contracts. The researcher believes the actual use of Preaward Survey information for this alternative 
purpose is lower than reflected in CAO interview responses. This opinion is based upon the low seven 
percent notification rate on overridden contracts reflected in Table 23 and CAO personnel concerns about 
limited internal Preaward Survey distribution within the DCMC. Additionally, it was noted that DCMC 
personnel were not aware of two of the nine overridden contracts discovered during this study. As such, no 
special surveillance or risk mitigation plans were developed for those contracts. 

In order to increase the use of Preaward Survey information as a risk mitigation tool during post- 
award planning and management, several Preaward Survey Managers recommended that Preaward 
Surveys be promoted more aggressively as part of DCMC’s “Early Contract Administration Services 
(CAS)” program. One part of this program focuses on the evaluation of contractors’ performance 
capabilities during source selection. This part of the Early CAS Program helps PCOs select capable 
contractors and identify potential performance risks. As a team, Procuring Activity and CAO personnel 
should use the information learned during both positive and negative Preaward Surveys to develop 
appropriate levels of surveillance and specific risk mitigation plans. 

Prior to making an award in the face of a Negative Preaward Survey, Procuring Activity and 
CAO personnel should discuss possible ways to alter contract terms and conditions to increase the 


probability of successful contract completion. Such mitigation actions might include the use of 
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performance bonds, progress payments, First Article Testing, frequent inspection points, liquidated 
damages, contract incentives and directed sources of material. When circumstances warrant, the team 
should establish a specific surveillance plan for each contractor who has received an overridden contract. 
This plan should address specific actions to mitigate the risks associated with each Negative Preaward 
Survey finding. If forced to award an overridden contract to a sole source contractor, Procuring Activity 
and CAO personnel should discuss whether the contractor’s performance history and current capabilities 
warrant parts breakout for competitive procurement or the need to qualify additional vendors. If the 
Preaward Survey findings are positive, Procuring Activity and CAO personnel should discuss the risks, if 
any, associated with reducing or elimination post-award surveillance. 

Given the above analysis, the researcher concludes that DOD Is currently not taking full 
advantage of Preaward Survey information as a source selection and risk management too]. The researcher 
believes that expanded use of Preaward Survey information during source selection and post-award 
contract management will improve the DOD acquisition process. 

F. SUMMARY 

Based upon the foregoing analysis, the researcher concludes that the Preaward Survey process is 
generally operating effectively but that improvements could be made to enhance its use as a source 
selection and risk management tool. The vast majority of Procuring Activity personnel believe that 
Preaward Surveys provide useful information for contractor responsibility determinations. As such, the 
current process helps ensure that only responsible and capable contractors receive Government contracts. 
Although effective in this role, the researcher concludes that Preaward Surveys may not always be useful to 
facilitate source selection decisions. Additionally, this study concludes that Preaward Surveys are 
underutilized as a risk management tool during preaward decisions and post-award contract management. 
Specific recommendations to enhance Preaward Surveys as a source selection and risk management tool 


are provided in Chapter V. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS 

The overall purpose of this study is to examine the Department of Defense’s (DOD’s) experience with 
contracts awarded in the face of a Negative Preaward Survey and assess the effectiveness of the Preaward 
Survey process as a source selection and risk management tool. Upon completion of the analysis presented 
in Chapter IV, the researcher concludes that the Preaward Survey process is operating at a reasonable level 
of efficiency and effectiveness, but that improvements could be made to enhance its use as a source 
selection and risk management tool. This conclusion is based upon findings that Preaward Survey 
information is being utilized effectively for contractor responsibility determinations. However, it was 
noted that Preaward Surveys are not always useful to facilitate source selection decisions and that Preaward 
Surveys are not being fully utilized as a risk management tool during preaward decisions and post-award 
contract management. The remainder of this Chapter provides a detailed summary of specific conclusions 
for the individual research questions, recommended changes to the Preaward Survey process and suggested 
areas of further study. 

The Defense Contract Management Command (DCMC) performs the most Preaward Surveys within 
DOD. In 1996, DCMC performed over 3400 Preaward Surveys on prospective Government contractors. 
The vast majority of these prospective contractors were small businesses vying for relatively low dollar 
value DOD contracts. From a sample of Preaward Surveys performed at one DCMC in 1995, the median 
dollar value of proposed contracts was approximately $127,000, with 47 percent of the contracts valued at 
less than the Simplified Acquisition Threshold ($100,000) specified in the Federal Acquisition Regulation. 
Approximately 23 percent of all Preaward Surveys performed by DCMC in 1996 resulted in a “No Award” 


recommendation. The primary reason for issuance of a Negative Preaward Survey was due to the 
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contractor’s production capability, followed by financial, quality and technical capabilities. Typically, 
these four factors account for over 90 percent of the Negative Preaward Surveys issued by DCMC. 

There is a general perception among Contract Administration Office (CAO) personnel that a relatively 
high percentage of DCMC’s “No Award” recommendations are overridden by Procuring Contracting 
Officers (PCOs) and that the vast majority of these decisions are not justified. During this study, Contract 
Administration Office (CAO) personnel believed that approximately 32 percent of all Negative Preaward 
Surveys are overridden by PCOs. Out of these awards, CAO personnel believe that just 38.5 percent of 
these are fully justified. The basis for this belief is primarily due to a perception that only a third of 
overridden contracts are completed satisfactorily. This study concludes that the percentage of contracts 
awarded in the face of a Negative Preaward Survey is much lower than CAO personnel perceptions. Out of 
68 Negative Preaward Surveys issued by DCMC Philadelphia in 1995, just nine or 13.2 percent of these 
were overridden by PCOs. This percentage is in line with Procuring Activity personnel perceptions that 
approximately 15 percent of Negative Preaward Surveys are overridden by PCOs. 

The most frequently cited reasons for overriding a Negative Preaward are “receipt of a Small Business 
Administration (SBA) Certificate of Competency (COC), “receipt of undated, favorable information about 
a contractor’s capabilities” and “knowledge that the contractor has performed satisfactorily on recent 
similar contracts.” Based upon analysis of these reasons presented in Chapter IV, the vast majority of 
overridden contracts are justified either by the particular circumstances surrounding the procurement or by 
SBA COC regulations. 

To assess contractor performance under overridden contracts, CAO and Procuring Activity personnel 
were asked to list the percentage of contracts completed satisfactorily under various contract award 
justifications. In response, CAO personnel stated that 37 percent of overridden contracts are completed 
satisfactorily, while Procuring Activity personnel stated that 75 percent are completed satisfactorily. This 
divergence in data makes it difficult to draw a definitive conclusion with regard to the overall percentage of 
overridden contracts which are competed satisfactorily. However, the data collected in this area indicate 


the relative risks associated with various override decisions. Both CAO and Procuring Activity personnel 
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believe that contracts awarded via SBA COCs and sole source justifications experience more performance 
problems and are less likely to be completed satisfactorily. Conversely, CAO and Procuring Activity 
personnel have the highest confidence in awards based upon the following reasons: “Negative Preaward 
Survey findings are believed to have little or no bearing on contract performance,” “Receipt of favorable, 
updated information about the contractor’s capabilities,” “Successful rebuttal by the contractor of 
Negative Survey findings,” and “The contractor has performed satisfactorily on recent, similar contracts.” 
Given the above findings, the researcher concludes that the greatest performance risks are associated with 
overridden contracts awarded under SBA COC and sole source justifications. To mitigate contract 
performance risks, DOD personnel should consider the relative risks associated with an override decision 
during the development of procurement strategies and post-award management plans. 

Universally, both CAO and Procuring Activity personnel voiced strong concerns about the SBA’s role 
within the DOD acquisition process and the adequacy of SBA’s COC award decisions. In particular, DOD 
personnel expressed frustration with SBA’s perceived lenient COC award policy, the lack of accountability 
for SBA’s actions, the apparent duplicative nature of the COC process, the timeliness of COC 
determinations, and the authority SBA possesses to override DCMC’s Negative Preaward Survey 
recommendations and PCO’s non-responsibility determinations. A large part of the frustration expressed 
by these respondents stems from the perception that less than half of the contracts awarded under COC 
circumstances are completed satisfactorily. Although this study did not collect data to substantiate CAO 
and Procuring Activity personnel perceptions, the researcher believes that the conduct of a second 
capability survey on a prospective contractor after DCMC has performed a Preaward Survey is duplicative. 
Also, as noted in this study, the current DCMC Preaward Survey process was judged to be operating 
effectively as a contractor responsibility determination tool. Therefore, there are no data to indicate that 
DCMC Preaward Surveys are biased against Small Businesses. As such, this study concludes that SBA’s 
COC program performs an unnecessary, duplicative function. 

Although, most CAO and Procuring Activity personnel felt that the Preaward Survey process is 


working effectively, many CAO personnel voiced concern about the lack of Preaward feedback on the 
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usefulness of Preaward Surveys and the specific award decisions made after issuance of a Preaward 
Survey. On average, Procuring Activities provide feedback on Positive Preaward Surveys less than 20 
percent of the time and notice of overridden contracts less than seven percent of the time. Additionally, 
when notice is provided to DCMC, many CAO personnel stated that they do not always receive a copy of 
Preaward Survey findings via internal DCMC distribution channels. The researcher believes that the lack 
of Preaward Survey feedback from Procuring Activities and limited distribution of Survey reports within 
the CAO, hampers DCMC’s ability to develop proactive risk mitigation plans on overridden contracts. 
Also, the information contained in Positive Preaward Surveys may not be used to tailor post-award 
surveillance plans. Given the above, this study concludes that the information contained in both Positive 
and Negative Preaward Surveys may be underutilized as a risk mitigation tool during post-award contract 
management. 

In addition to the lack of feedback on Preaward Surveys, CAO and Procuring Activity personnel noted 
that improvements could be made in how Preaward Survey information is communicated to users. As 
pointed out by several respondents, the value of a Preaward Survey report is limited substantially, if the 
findings are not disseminated to all potential users. To improve the current communication, respondents 
recommended DCMC encourage greater participation in the conduct of Preaward Surveys and greater use 
of Post-Award Orientation Conferences on known or “high risk” contractors . To increase the availability 
and sharing of Preaward Survey data, several Procuring Activity personnel recommended creation of a 
centralized Preaward Survey data base which could be accessed via the Internet. Based upon these 
comments, this study concludes that improvements could be made to improve the accessibility and 
dissemination of Preaward Survey information. 

During the conduct of this study, many Procuring Activity personnel expressed concern about the 
timeliness of Preaward Surveys. Several PCOs stated that they do not always request a Preaward Survey 
because the turn-around time does not meet their needs. Additionally, Procuring Activity personnel stated 
that DCAA needed to improve the turn-around time on Preaward Survey accounting system evaluations. 


Conversely, CAO personnel stated that they needed more time than the seven working days specified in the 
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FAR to conduct a high quality Preaward Survey, and that they should be given additional time to perform 
the Survey [Ref. 115:p. 9-4]. The researcher believes both concerns presented above are valid from each 
presenter’s viewpoint. However, given the perspective that the customer defines the end product and 
specifies the due date, this study concludes that Preaward Survey turn-around does not always meet the 
needs of DCMC’s customers. Accordingly, DCMC should aggressively pursue ways to improve the 
timeliness of Preaward Surveys and seek alternative ways to satisfy customer requests for contractor 
capability information. 

During the interview potion of this study, DOD personnel provided numerous comments and 
recommended changes to improve the content of Preaward Surveys. One common theme emerged from 
these interviews. Procuring Activity personnel thought that Preaward Surveys were a useful tool to 
facilitate contractor responsibility determinations but that Preaward Surveys were not always useful for 
source selection and best value decisions. Most Negative Preaward Surveys and some Positive Preaward 
Surveys are used by PCOs to exclude non-responsible or marginal contractors from the pool of contractors 
being considered for award. However, Preaward Surveys could not always be used to help determine 
which contractor to select in a best value procurement. To enhance the usefulness of Preaward Surveys 
during source selection, Procuring Activity personnel recommended that Preaward Surveys contain more 
information on past performance, noted trends, ongoing corrective actions and risk assessments. 
Additionally, PCOs and Buyers wanted DCMC to provide more analysis of the data contained in the 
Preaward Survey, vice just presenting data and facts. CAO personnel recommended that PCOs assign a 
higher weight to Preaward Survey findings, particularly delivery performance and past performance 
history, during source selections. Given the above, this study concludes that Preaward Surveys are not 
always useful during source selection. The researcher believes that many of the recommendations 
presented above have the potential to enhance Preaward Surveys as a source selection tool during best 
value procurements. 

In general, CAO and Procuring Activity personnel believe that Preaward Surveys are useful for post- 


award planning and management. The majority of Procuring Activity personnel stated that they primarily 
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use Preaward Survey information to determine the level of post-award surveillance required of DCMC. 
However, a relatively few number of PCOQs stated that they use this information to mitigate performance 
risk via changed contract terms or conditions. As expected, CAO personnel stated that Preaward Survey 
information was very useful during post-award contract management. When asked how they use this 
information, the majority of CAO personnel stated that this information was used to develop post-award 
surveillance plans. This study did not conduct follow-up research of contract files to determine if Preaward 
Survey information was actually used to change contact terms and/or develop surveillance and risk 
mitigation plans on overridden contracts. However, the researcher believes the use of Preaward Surveys 
for these alternate purposes is much lower than indicated by CAO and Procuring Activity responses. This 
belief is based upon the reported low Procuring Activity use of Preaward Survey information after contract 
award, the low notification rate on overridden contracts and the limited internal distribution of Preaward 
Surveys within DCMC. Accordingly, the researcher concludes that DOD is not taking full advantage of 
Preaward Survey information as a source selection and risk management tool. 
B. RECOMMENDATIONS 

The following recommendations are provided to improve the efficiency and effectiveness of the 
Preaward Survey process. Additionally, the researcher believes that these changes will also enhance the 
Preaward Survey process as a source selection and risk management tool. These recommendations are 


based upon the analysis contained in Chapter IV and the conclusions presented in this Chapter. 


1. DCMC should limit the number of Preaward Surveys performed on low dollar value contracts to 
only those required in support of a critical mission or application. 


All non-critical requests for contractor capability information with a proposed contract value 
below $100,000 should be accomplished via alternate Government sources or commercially available 
sources. As noted in this study, almost half of Preaward Surveys conducted by one DCMC in 1996 
involved contracts valued at less than the $100,000. Adoption of this recommendation will significantly 
reduce the number of Preaward Surveys performed by DCMC and allow DCMC to reallocate scarce 


resources to higher priority requirement. As noted during in a 1995 DLA Contract Administration Services 


122 


Process Action Team Report, Procuring Activities can often obtain contractor responsibly information via 


commercially available sources more quickly than via a formal on-site Preaward Survey. 


2. DCMC and its customers should seek mutually beneficial ways to improve the timelines of 
Preaward Surveys. 


One way to improve turn-around is to encourage the tailoring of Preaward Survey requests. This 
helps DCMC focus on customer requirements and helps ensures that effort is not wasted collecting 
information that is not needed for contractor responsibility determinations or source selection decisions. In 
addition to encouraging the use of commercially available sources of contractor capability information, 
DCMC should investigate alternative ways to satisfy customer requests for contractor capability 


information. 


3. To improve feedback on contract award decisions after issuance of a Preaward Survey, emphasis 
should be renewed on placing the associated Preaward Survey Serial number on the face of the contract, 
and when a contract 1s awarded in the face of a Negative Preaward Survey, the Procuring Activity should 
notify the Preaward Survey Manager (PASM) of the contract award and award justification. 


This notification should occur shortly after the award and should be made via the most 
expeditious means possible. The PASM should then ensure that all cognizant CAO team members are 
provided a copy or have ready access to Negative (and Positive) Preaward Survey findings. These 
recommendations should help facilitate proactive development of appropriate contract surveillance and risk 
mitigation plans. 


4. To facilitate increased use of Preaward Survey findings, DCMC should aggressively promote 
Preaward Surveys as part of their “Early Contract Administration Services (Early CAS)” Program. 


This program ensures that PCOs have the information they need to select capable contractors and 
helps identify potential performance risks. As such, Preaward Surveys are an integral component of 
DCMC’s Early CAS Program. In addition to using Early CAS, CAO and Procuring Activities should 
increase their use of Post-Award Orientation Conferences on known or suspected “high” risk contractors 
who have received marginal ratings or a “No Award” recommendation. The information contained in 
Preaward Surveys should be used as points of discussion during these conferences. On all high risk 


overridden contracts or contracts requiring additional management attention, the CAO should be required 


to have a written surveillance and mitigation plan which addresses all areas of concern contained in a 


Preaward Survey. 


5. To facilitate expanded use of Preaward Surveys during best value procurements, DCMC should 
include more information on past performance, noted trends, ongoing corrective actions and risk 


assessments. 

Additionally, DCMC personnel should be given feedback that PCOs and Buyers desire more 
analysis of the data contained in the Preaward Survey, vice just presenting data and facts. Depending upon 
the procurement strategy, Procuring Activities should consider applying a higher weighting to Preaward 
Survey information, such as delivery performance and past performance history, during source selections. 
These actions will enhance the usefulness of Preaward Surveys as a source selection tool, especially during 


best value procurements. 


6. Procuring Activities should expand their use of Preaward Survey information during pre-award 
contract planning and management. 


Prior to awarding a contract in the face of a Negative Preaward Survey, PCOs should review the 
Negative Survey findings with CAO personnel to determine if the contract terms and conditions could be 
altered to mitigate performance risk. For instance, the particular procurement situation and Negative 
Preaward Survey findings may warrant inclusion of performance bonds, progress payments, First Article 
Testing, frequent inspection points, liquidated damages, contract incentives and directed sources of 
material. Proactive planning prior to contract award may reduce post-award performance problems and 


overall acquisition costs. 


7. The SBA COC Program should be disestablish in its entirety. This program duplicates efforts 
performed by DCMC. 


The SBA COC Program costs taxpayers an enormous amount of money to perform this mission 
with results that often result in additional outlays of time, money and resources. In this era of limited 
resources and budget constraints, the Government can ill afford the cost of duplicative programs. In lieu of 
performing COC determinations, SBA should focus on ways to improve a perspective contractor’s 


capability to a level which makes them eligible to compete on future awards. 
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8. Ifthe SBA COC Program is not disestablish, SBA should be encouraged _to participate in DCMC 
Preaward Surveys performed on known or suspected “high risk” contractors. 


Non-traditional Preaward Survey participants within the CAO team should also be encouraged to 
participate in these surveys. The SBA representatives could either act as observers or team members to 
facilitate a concurrent SBA review on prospective small business contractors. This would improve the 
turn-around time on SBA COC decisions and could foster improved communication between SBA and 


DCMC. 


9. To increase the accountability of SBA COC decisions, SBA should be required to discuss its 
rationale with DCMC prior to issuance of a COC. 


This gives DCMC an opportunity to assess SBA’s rationale for overriding DCMC’s “No Award” 
recommendation. It also provides DCMC an opportunity to provide additional information or clarification 
of DCMC’s concerns. If SBA does issue a COC, SBA should be required to furnish COC findings and its 
rationale to the cognizant CAO and Procuring Activity. Requiring SBA to forward COC findings and 
associated justifications could instill greater accountability into the COC process. As with Negative 
Preaward Survey information, COC findings could be useful during post-award planning and contract 
administration. As another means to increase the integrity of the acquisition process, DOD should 


aggressively pursue the right to override a COC determination on its own requirements. . 


10. To facilitate communication of Preaward Survey data, DOD should investigate the feasibility and 
desirability of establishing a centralized Preaward Survey database which could be accessed via the 


Internet. 

An accessible database may shorten the time it takes to make a responsibility determination, 
decrease the number of Preaward Surveys performed by DCMC and improve the efficiency of the 
Preaward Survey process. 

C. ANSWERS TO THE RESEARCH QUESTIONS 
This section provides answers to the study’s primary and seven subsidiary research questions. The 
answers are based upon the analysis contained in Chapter IV and the conclusions presented in this Chapter. 


The subsidiary research questions support the answer to the primary research question. As such, answers 
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to the seven subsidiary questions will be presented first, followed by the answer to the primary research 
question. 

1. What is a Preaward Survey and how are Preaward Surveys used as a source selection tool? 

Preaward Surveys are a method used to determine a prospective contractor’s capabilities to 

perform on a proposed Government contract. These Surveys provide PCOs with useful information about a 
contractor’s production, technical, management, quality assurance, financial, and overall business 
capabilities. When the Survey discovers negative or marginal findings about a contractor’s capabilities, the 
PCO can elect to use this information to exclude that contractor from the pool of contractors being 
considered for contract award. In some cases, Preaward Survey information, such as past performance 
history and risk assessments, may be used to facilitate a ranking of the contractor’s capabilities during a 


“best value” procurement strategy. 


2. What isa Negative Preaward Survey and what are the various DOD policies with respect to 
awarding a contract in the face of a Negative Preaward Survey? 


A Negative Preaward Survey is a report which contains negative findings about a contractor’s 
capabilities to perform on a proposed Government contract. In these cases, DCMC found the prospective 
contractor non-responsible and issued a “No Award” recommendation to the Procuring Activity. The 
Federal Acquisition Regulation (FAR) and various Service regulations do not prohibit a PCO from 
overriding a Negative Preaward Survey. However, these regulations require that the PCO make a 
responsibility determination prior to award of a Government contract and that the PCO document the 
contract award file to that effect. In some cases, local regulations or procedures may require management 


review at a level higher than the PCO. 
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3. What are the relative percentages of positive and negative Preaward Survey recommendations? 


In 1996, DCMC conducted over 3,400 Preaward Surveys on prospective Government contractors. 
Approximately 77 percent of these Surveys were positive, while 23 percent were negative. These 
percentages represent an average across DCMC’s three districts. Some of the larger, non-resident DCMCs 


reported Negative Preaward Survey rates as high as 42 percent. 
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4. What are the primary causes of Negative Preaward Survey recommendations and what impact do 
they have on contract performance? 


The majority of Negative Preaward Survey recommendations were caused by deficiencies in a 
contractor’s production, financial and quality assurance capabilities. Under these circumstances, DCMC 
usually noted poor production planning, unsatisfactory delivery performance, demonstrated quality 
problems or lack of a quality assurance system, and inadequate financial resources. If these deficiencies 
were not corrected at time of award or shortly thereafter, there is a high probability that the contractor 


would experience the same or similar performance problems on the instant contract. 


5. How closely do the reasons for contract performance failures line up with associated negative 
Preaward Survey recommendations? 


This question assumes that a PCO has reviewed all the circumstances surrounding the particular 
procurement and Negative Preaward Survey findings and has decided to award a contract in the face of a 
Negative Preaward Survey. The correlation of a contractor’s performance on an overridden contract 
depends largely upon the circumstances surrounding the award and the PCO’s decision for overriding 
DCMC’s “No Award” recommendation. There is a high correlation between negative performance and 
Negative Preaward Survey findings when the award is based upon an SBA COC decision or sole source 
justification. However, there is a greater probability that the contract will be completed satisfactorily when 
the PCO bases the award upon the following override justifications: (1) the contractor has performed 
satisfactorily on recent, similar contracts, (2) the PCO received updated, favorable information about the 
contractor’s capabilities, (3) the contractor has successfully rebutted Negative Preaward Survey findings or 
corrected noted deficiencies, and (4) the reported deficiencies are deemed not to have a material impact on 


the contractor’s ability to perform the proposed contract. 


6. Do any ofthe reasons for awarding a contract in the face of a negative Preaward Survey play a 
role in a contractor’s satisfactory or unsatisfactory performance? 


As discussed in the response to question number five above, the circumstances surrounding the 
particular procurement and the associated override justification may mitigate some of the risks associated 


with overridden contracts. For example, knowledge that the prospective contractor has recently received a 
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letter of credit from a bank may mitigate deficit working capital concerns noted in a Negative Preaward 
Survey. Likewise, knowledge that the contractor has recently supplied the same or similar item to another 


Government activity or commercial customer may mitigate the performance risk on the proposed contract. 


7. What changes to the use of Preaward Surveys are needed to enhance it as a part of the source 
selection process? 


Currently, Preaward Surveys are a useful tool to facilitate contractor responsibility determinations. 
To enhance its effectiveness during source selection, DCMC should include more information on past 
performance, noted trends, ongoing corrective actions, and risk assessments. Additionally, Procuring 
Activities should consider assigning Preaward Survey information, such as delivery performance and past 


performance history, a higher weighting during source selections. 


8. What is DOD’s experience with contracts awarded in the face of a negative Preaward Survey 
recommendation and how might this information be used to improve the source selection process? 


This study concludes that DCMC issues a Negative Preaward Survey approximately 25 percent of 
the time and that PCOs override DCMC’s “No Award” recommendation approximately 15 percent of the 
time. The vast majority of contracts awarded in the face of a Negative Preaward Survey appear to be 
justified based upon either SBA COC regulation or the specific circumstances surrounding the 
procurement. Based upon a review of the decisions provided for overriding Negative Preaward Surveys, it 
appears that PCOs are weighing all relevant factors prior to making a responsibility determination and 
subsequent award decision. PCOs have a high degree of confidence in the vast majority of award decisions 
under their control, but have a low degree of confidence in SBA COC decisions and sole source awards. 
This concern is validated in subsequent contract performance. Contracts awarded under SBA COC 
decisions and sole source awards experience more performance problems. Conversely, contract awards 
based upon knowledge of recent satisfactory performance or updated, favorable information about a 
contractor’s capabilities have less performance risk. 

In light of the above findings, DOD should tailor its procurement strategies and preaward risk 
mitigation plans based upon the relative risks associated with various override justifications. Additionally, 


greater emphasis should be placed on changing contract terms and conditions, as necessary, to mitigate the 
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risks associated with overridden contracts. As warranted, Procuring Activities should expand the use of 
best value procurement strategies and should alter future procurement strategies based upon lessons learned 
under overridden contracts. 

D. AREAS OF FURTHER RESEARCH 

As noted in Chapter I, the scope of this study was limited to specific research questions concerning 
DOD’s experience with contracts awarded in the face of a Negative Preaward Survey. As such, this study 
only covered a portion of the issues surrounding the DCMC Preaward Survey process. The following areas 
are recommended for further research: 

1. Evaluate SBA COC procedures to ensure the adequacy of COC decisions. To facilitate this study, 
the researcher should evaluate the relative percentages of COCs issued versus the number of COCs 
requested but declined, as well as the associated reasons for issuing or denying a COC. Additionally, data 
on contractor performance should be evaluated to determine if there are greater performance risks 
associated with various COC justifications. 

2. Evaluate the cost effectiveness of maintaining a Preaward Survey capability and associated 
Preaward Survey management staff at DCMCs which perform a relatively low number of Preaward 
Surveys on a monthly basis. Assess whether the Preaward Survey function at these smaller DCMCs could 
be assumed by a surrounding, larger DCMC. Determine the pros and cons of such a transfer of function. 

3. Compare DOD’s Preaward Survey process with industry source selection and responsibility 
determination procedures to determine if any industry practices could be adopted by DOD. 

4. Conduct a study to determine the universe of commercially available products which could satisfy 
a customer’s request for contractor responsibility information and assess the adequacy of each potential 
alternative. 

5. Assess the feasibility of developing a centralized Preaward Survey database which could be 
accessed via the Internet. Determine the pros and cons associated with such a database. Compare the 
advantages and disadvantages of a DOD developed database against a system which relies solely upon 


commercially available contractor information sources. 
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APPENDIX A 


INTERVIEW QUESTIONS FOR CONTRACT ADMINISTRATION OFFICE PERSONNEL 


1. What is your current position? (Please circle the appropriate letter). 


a. Administrative Contracting Officer (ACO) 

b. Contract Administrator or Contract Specialist 
c. Industrial Specialist 

d. Quality Assurance Specialist 

e. Cost/Price Analyst 

f. Other (please list): 


2. How many years of experience do you have in government contract administration? (Please circle the 
appropriate letter). 


Less than 2 years 
2 to 5 years 

5 to 10 years 
Over 10 years 


Anoop 


What percentage of time do Procuring Activities notify you or your organization when they award a 
contract in the face of a negative Preaward Survey recommendation? (I.e., The PCO overrides a “No 
Award” recommendation and awards a contract to that contractor anyway). (Please circle one of the 

percentages below and provide any desired comments in the space provided). 


aS) 


NEVER ALWAYS 
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


Comments: 


4 Based upon your experience, how frequently do Procuring Activities award a contract in the face of a 
negative Preaward Survey recommendation? (Please circle one percentage below). 


NEVER ALWAYS 
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


Comments: 
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5. Do you feel that the awarding of contracts in the face of a negative Preaward Survey are properly 
justified by the Procuring Activity? (Please annotate the percent you feel are justified). 


NEVER ALWAYS 


ae ee ee eee eee prin 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


Comments: 


6. What is the MOST frequent reason PCOs give for awarding a contract in the face of a negative 
Preaward Survey? (Please circle only one choice). 


No other source/sole source contractor 

Contractor has performed satisfactorily on most recent contract(s) 

Marginal financial condition can be mitigated with Progress Payments 

Meeting delivery schedule is not critical 

PCO willing to accept reduced quality to meet urgent service requirement 

SBA issued a Certificate of Competency (COC) 

PCO received updated information about the contractor’s capabilities 

Contractor successfully rebutted negative Preaward Survey findings 

Reported deficiencies won’t impact contractor’s ability to perform the proposed contract 
Other (Please List) 


Tm rg mono pS 


7. Based upon your response to question number 6 above, what percentage of contracts are completed 
satisfactorily (no or minimal problems noted) under these circumstances? (Please circle only one 
percentage). 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


Comments: 


8. What is the SECOND MOST frequent reason PCOs give for awarding a contract in the face of a 
negative Preaward Survey? (Please circle only one choice). 


No other source/sole source contractor 

Contractor has performed satisfactorily on most recent contract(s) 

Marginal financial condition can be mitigated with Progress Payments 

Meeting delivery schedule is not critical 

PCO willing to accept reduced quality to meet urgent service requirement 

SBA issued a Certificate of Competency (COC) 

PCO received updated information about the contractor’s capabilities 

Contractor successfully rebutted negative Preaward Survey findings 

Reported deficiencies won’t impact contractor’s ability to perform the proposed contract 
Other (Please List) 
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9. Based upon your response to question number 8 above, what percentage of contracts are completed 
satisfactorily (no or minimal problems noted) under these circumstances? (Please circle only one 
percentage). 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


Comments: 


10. What is the THIRD MOST frequent reason PCOs give for awarding a contract in the face of a 
negative Preaward Survey? (Please circle only one choice). 


No other source/sole source contractor 

Contractor has performed satisfactorily on most recent contract(s) 

Marginal financial condition can be mitigated with Progress Payments 

Meeting delivery schedule is not critical 

PCO willing to accept reduced quality to meet urgent service requirement 

SBA issued a Certificate of Competency (COC) 

PCO received updated information about the contractor’s capabilities 

Contractor successfully rebutted negative Preaward Survey findings 

Reported deficiencies won’t impact contractor’s ability to perform the proposed contract 
Other (Please List) 
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11. Based upon your response to question number 10 above, what percentage of contracts are completed 
satisfactorily (no or minimal problems noted) under these circumstances? (Please circle only one 
percentage). 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


Comments: 


12. Please rank the FIVE most common reasons why contractors receive a negative Preaward Survey 
recommendation. (1 being the most common reason, 2 being the second most common reason, etc.). 


REASON RANK REASON RANK 
Technical Govt Property Control 

Production Transportation 

Quality Assurance Packaging 

Financial Security Clearance 

Accounting System Plant Safety 

Environmental Non-Responsive 

Other: 


13. When a contractor’s performance on a contract awarded in the face of a negative Preaward Survey is 
marginal or unsatisfactory, how frequently do the contractor’s performance failures or deficiencies line 
up with associated negative Preaward Survey findings? 


NEVER | ALWAYS 
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


Comments: 


14. Ifa contractor is awarded a contract in the face of a negative Preaward Survey recommendation, do 
you find this information useful to you during post-award contract administration? (Please annotate on 
a scale of 0 to 10 the usefulness of this information with 0 signifying “No Use” and 10 signifying 


“Very Useful’). 


NO USE | | | | | | VERY USEFUL 
0 ] Z 3 4 =) 6 7 8 9 10 
Comments: 


15. Considering your responses to question 14, please provide specific examples of how you use negative 
Preaward Survey information during post-award contract administration. 


Examples: 


16. What changes to the Preaward Survey process would you recommend to enhance its effectiveness as a 
source selection tool? 


Recommendations: 
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17. What changes to the Preaward Survey process would you recommend to enhance its effectiveness as a 
risk management tool during post-award contract administration? 


Recommendations: 


18. Please feel free to write down any additional comments you may have about the Preaward Survey 
process. 


General Comments: 


G2 


APPENDIX B 


INTERVIEW QUESTIONS FOR PROCURING ACTIVITY PERSONNEL 


What is your current position? (Please circle the appropriate letter). 


a. Procuring Contracting Officer (PCO) 
b. Contracts Manager 

c. Contract Specialist 

d. Buyer 

e. Procurement Analyst 

f. Other (please list): 


How many years of experience do you have in government acquisition? (Please circle the appropriate 
letter). 


a. Less than 2 years 
b. 2to5 years 

c. 5 to 10 years 

d. 


Over 10 years 


Do you find the information contained in Preaward Surveys useful during source selection? (Please 
annotate on a scale of 0 to 10 the usefulness of this information with 0 signifying “No Use” and 10 
signifying “Very Useful’). 


NO USE | | | | | | VERY USEFUL 
0 ] 2 5 4 5 6 i 8 9 10 
Comments: 


How frequently do you or your organization award a contract in the face of a negative Preaward 
Survey recommendation? (I.e., The PCO overrides a “No Award” recommendation and awards a 
contract to that contractor anyway). Please circle one percentage below. 


NEVER ALWAYS 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


Comments: 
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5. What percentage of time do you or your organization notify Contract Administration Offices (CAO) of 
contracts awarded in the face of a negative Preaward Survey recommendation? (Please circle one of the 
percentages below and provide any desired comments in the space provided). 


NEVER ALWAYS 
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


Comments: 


6. Do you feel that the awarding of contracts in the face of a negative Preaward Survey are properly 
justified by you or your organization? (Please annotate the percent you feel are justified). 


NEVER ALWAYS 
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


Comments: 


7. What is the MOST frequent reason you or your organization give for awarding a contract in the face 
of a negative Preaward Survey? (Please circle only one choice). 


No other source/sole source contractor 

Contractor has performed satisfactorily on most recent contract(s) 

Marginal financial condition can be mitigated with Progress Payments 

Meeting delivery schedule is not critical 

PCO willing to accept reduced quality to meet urgent service requirement 

SBA issued a Certificate of Competency (COC) 

PCO received updated information about the contractor’s capabilities 

Contractor successfully rebutted negative Preaward Survey findings 

Reported deficiencies won’t impact contractor’s ability to perform the proposed contract 
Other (Please List) 


7 eS do o>) 0 eo = 


8. Based upon your response to question number 7 above, what percentage of contracts are completed 
satisfactorily (no or minimal problems noted) under these circumstances? (Please circle only one 
percentage). 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


Comments: 
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10. 


ee 


What is the SECOND MOST frequent reason you or your organization give for awarding a contract 
in the face of a negative Preaward Survey? (Please circle only one choice). 


No other source/sole source contractor 

Contractor has performed satisfactorily on most recent contract(s) 

Marginal financial condition can be mitigated with Progress Payments 

Meeting delivery schedule is not critical 

PCO willing to accept reduced quality to meet urgent service requirement 

SBA issued a Certificate of Competency (COC) 

PCO received updated information about the contractor’s capabilities 

Contractor successfully rebutted negative Preaward Survey findings 

Reported deficiencies won’t impact contractor’s ability to perform the proposed contract 
Other (Please List) 


Sor aga hoe Bo op 


Based upon your response to question number 9 above, what percentage of contracts are completed 
satisfactorily (no or minimal problems noted) under these circumstances? (Please circle only one 
percentage). 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


Comments: 


. What is the THIRD MOST frequent reason you or your organization give for awarding a contract in 


the face of a negative Preaward Survey? (Please circle only one choice). 


No other source/sole source contractor 

Contractor has performed satisfactorily on most recent contract(s) 

Marginal financial condition can be mitigated with Progress Payments 

Meeting delivery schedule is not critical 

PCO willing to accept reduced quality to meet urgent service requirement 

SBA issued a Certificate of Competency (COC) 

PCO received updated information about the contractor’s capabilities 

Contractor successfully rebutted negative Preaward Survey findings 

Reported deficiencies won’t impact contractor’s ability to perform the proposed contract 
Other (Please List) 


ao) St eo Ome 


Based upon your response to question number |1 above, what percentage of contracts are completed 
satisfactorily (no or minimal problems noted) under these circumstances? (Please circle only one 
percentage). 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


Comments: 


13. Please rank the FIVE most common reasons why contractors receive a negative Preaward Survey 
recommendation. (1 being the most common reason, 2 being the second most common reason, etc.). 


REASON RANK REASON RANK 
Technical Govt Property Control 
Production Transportation 

Quality Assurance Packaging 

Financial Security Clearance 

Accounting System Plant Safety 

Environmental Non-Responsive 

Other: 

Comments: 


14. When a contractor’s performance on a contract awarded in the face of a negative Preaward Survey is 
marginal or unsatisfactory, how frequently do the contractor’s performance failures or deficiencies line 
up with associated negative Preaward Survey findings? 


NEVER emai 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


Comments: 


15. Do you find the information contained in negative Preaward Surveys useful for post-contract award 
planning and decisions? (Please annotate on ascale of 0 to 10 the usefulness of this information with 
0 signifying “No Use” and 10 signifying “Very Useful”). 





NO USE | | | | | | VERY USEFUL 
0 l 2 3 4 5 6 iy 8 9 10 
Comments: 


16. Considering your responses to questions 15 above, please provide specific examples of how you use 
negative Preaward Survey information in post-contract award planning and decisions. 


Examples: 


17. How frequently do you use positive Preaward Survey information in post-contract award planning and 
decisions? 


i ee | | 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


Comments: 


18. Considering your response to question 17 above, please provide specific examples of how you use 
positive Preaward Survey information in post-contract award planning and decisions. 


Examples: 


19. What changes to the Preaward Survey process would you recommend to enhance its effectiveness as a 
source selection tool? 


Recommendations: 


20. What changes to the Preaward Survey process would you recommend to enhance its effectiveness as a 
risk management tool during post-award contract administration? 


Recommendations: 


le? 


ACO 


CAO 


COLS 


DCAA 


DCMC 


DCMD 


DFARS 


DLA 


DOD 


FAR 


GAO 


PALT 


PAS 


PASM 


PASRB 


PCO 


PROCAS 


SBA 


APPENDIX C 


GLOSSARY OF ACRONYMS 
Administrative Contracting Officer 
Contract Administration Office 
Certificate of Competency 
Defense Contract Audit Agency 
Defense Contract Management Command 
Defense Contract Management District 
DOD Federal Acquisition Regulation Supplement 
Defense Logistics Agency 
Department of Defense 
Federal Acquisition Regulation 
General Accounting Office 
Procurement Administrative Lead Time 
Preaward Survey 
Preaward Survey Manager 
Preaward Survey Review Board 
Procuring Contracting Officer 
Process Oriented Contract Administration Services 


Small Business Administration 
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